Attention in Deep Learning (Sommersemester 2021)
Lecturer: Prof. Dr. Christoph Lippert
(Digital Health - Machine Learning)
- Weekly Hours: 2
- Credits: 4
- Enrolment Deadline: 18.03.2021 - 09.04.2021
- Teaching Form: Seminar
- Enrolment Type: Compulsory Elective Module
- Course Language: English
Programs & Modules
- DATA-Konzepte und Methoden
- DATA-Techniken und Werkzeuge
Attention mechanisms - focusing on the important aspects of data while ignoring less relevant aspects - have been at the forefront of many recent breakthroughs in deep learning. Originally developed for language models, attention models have been adapted to vision, audio & music, graphs, and other data types.
We will read and discuss a number of landmark papers in the area, including recent developments.
Familiarity with deep learning, e.g. from the Deep Learning lecture by Prof. Lippert, or a similar course.
Basic understanding of RNNs such as LSTMs and GRUs, although there will be a short recap in the beginning.
Students are not required to know anything about attention mechanisms before the start of the seminar.
The course is also open for non-HPI students.
This seminar will have a format similar to a reading club/journal club, featuring student presentations and longer discussions. Each week, we will cover one paper related to attention mechanisms in deep learning, starting from the very basics and covering different application areas.
70% of the grade will be based on the paper presentation each student is required to give. Another 30% will be based on active participation in the weekly discussions.
The course starts on April 14th with a kick-off meeting.
Dates: Wednesdays, 13:30 - 15:00 via Zoom.
Meeting ID: 620 4705 2575