Attention in Deep Learning (Sommersemester 2021)
Dozent:
Prof. Dr. Christoph Lippert
(Digital Health - Machine Learning)
Allgemeine Information
- Semesterwochenstunden: 2
- ECTS: 4
- Benotet:
Ja
- Einschreibefrist: 18.03.2021 - 09.04.2021
- Lehrform: Seminar
- Belegungsart: Wahlpflichtmodul
- Lehrsprache: Englisch
Studiengänge, Modulgruppen & Module
- SCAD: Scalable Computing and Algorithms for Digital Health
- HPI-SCAD-C Concepts and Methods
- SCAD: Scalable Computing and Algorithms for Digital Health
- HPI-SCAD-T Technologies and Tools
- SCAD: Scalable Computing and Algorithms for Digital Health
- HPI-SCAD-S Specialization
- APAD: Acquisition, Processing and Analysis of Health Data
- HPI-APAD-C Concepts and Methods
- APAD: Acquisition, Processing and Analysis of Health Data
- HPI-APAD-T Technologies and Tools
- APAD: Acquisition, Processing and Analysis of Health Data
- HPI-APAD-S Specialization
- OSIS: Operating Systems & Information Systems Technology
- HPI-OSIS-K Konzepte und Methoden
- OSIS: Operating Systems & Information Systems Technology
- HPI-OSIS-T Techniken und Werkzeuge
- OSIS: Operating Systems & Information Systems Technology
- HPI-OSIS-S Spezialisierung
- DATA: Data Analytics
- HPI-DATA-K Konzepte und Methoden
- DATA: Data Analytics
- HPI-DATA-T Techniken und Werkzeuge
- DATA: Data Analytics
- HPI-DATA-S Spezialisierung
Beschreibung
Attention mechanisms - focusing on the important aspects of data while ignoring less relevant aspects - have been at the forefront of many recent breakthroughs in deep learning. Originally developed for language models, attention models have been adapted to vision, audio & music, graphs, and other data types.
We will read and discuss a number of landmark papers in the area, including recent developments.
Voraussetzungen
Familiarity with deep learning, e.g. from the Deep Learning lecture by Prof. Lippert, or a similar course.
Basic understanding of RNNs such as LSTMs and GRUs, although there will be a short recap in the beginning.
Students are not required to know anything about attention mechanisms before the start of the seminar.
The course is also open for non-HPI students.
Lern- und Lehrformen
This seminar will have a format similar to a reading club/journal club, featuring student presentations and longer discussions. Each week, we will cover one paper related to attention mechanisms in deep learning, starting from the very basics and covering different application areas.
Leistungserfassung
70% of the grade will be based on the paper presentation each student is required to give. Another 30% will be based on active participation in the weekly discussions.
Termine
The course starts on April 14th with a kick-off meeting.
Dates: Wednesdays, 13:30 - 15:00 via Zoom.
Zoom link:
uni-potsdam.zoom.us/j/62047052575
Meeting ID: 620 4705 2575
Passcode: 12923653
Zurück
