Tagging and Captioning Art-Historical Photographs (Wintersemester 2022/2023)

Dozent: Prof. Dr. Felix Naumann (Information Systems) , Alejandro Sierra Múnera (Information Systems) , Hendrik Rätz (Internet-Technologien und -Systeme) , Jona Otholt (Internet-Technologien und -Systeme)
Website zum Kurs: https://hpi.de/naumann/teaching/current-courses/ws-22-23/tagging-and-captioning-art-historical-photographs.html

Allgemeine Information

• Semesterwochenstunden: 4
• ECTS: 6
• Benotet: Ja
• Einschreibefrist: 01.10.2022 - 31.10.2022
• Prüfungszeitpunkt §9 (4) BAMA-O: 13.12.2022
• Lehrform: Projekt / Seminar
• Belegungsart: Wahlpflichtmodul
• Lehrsprache: Englisch
• Maximale Teilnehmerzahl: 12

Studiengänge, Modulgruppen & Module

Beschreibung

For centuries, the art world was a completely analog domain, but especially in recent years efforts were made to digitize artworks in order to conserve them and make them available to a wider audience. At the same time, this also enables the application of machine learning techniques such as image captioning/tagging to enrich large amounts of digitized art with meta information, such as tags or captions.

Using generated tags enables more efficient handling of the data for archivists and researchers because they can filter the data by categories, such as portraits, photos of architecture, etc. Additionally, it is a first step to make digitally-published artworks accessible to visually impaired people. They can already get an idea of the visual contents even when no captions (produced by a dedicated image captioning algorithm) are available.

 

Understanding and tagging images is already widely researched on natural images (e.g. ImageNet). However, there exist some differences to the data of the art domain regarding visuals and semantics (especially when it comes to paintings). Archives can suffer from image degradation over time and often. Artistic images can contain visual cues which differ from the typical objects depicted by figurative images. This is especially true for more abstract artworks.
In addition, art datasets often do not provide the labels needed to apply supervised techniques. 

 

In this seminar, we want to:

• tag and caption a real-life photograph archive provided by our project partner, the Wildenstein Plattner Institute,
• research ways how existing methods can be applied to our unlabeled data,
• study connections between image tagging and captioning, and
• investigate how we can bridge the domain gap between artistic and natural images.

Voraussetzungen

Basic knowledge on neural networks

Leistungserfassung

• Intermediate and final presentation
• Demonstration and report of method implementation and its experimental results

Termine

• 18th Oct. Presentation of the seminar
• 25th Oct. Introduction
• Weekly discussion
• 31st Registration period ends
• Intermediate presentation. (13th of December)
• Final presentation (31st January)
• Written report (28th of February)

Zurück