Hasso-Plattner-Institut25 Jahre HPI
Hasso-Plattner-Institut25 Jahre HPI

Attention in Deep Learning (Sommersemester 2021)

Dozent: Prof. Dr. Christoph Lippert (Digital Health - Machine Learning)

Allgemeine Information

  • Semesterwochenstunden: 2
  • ECTS: 4
  • Benotet: Ja
  • Einschreibefrist: 18.03.2021 - 09.04.2021
  • Lehrform: Seminar
  • Belegungsart: Wahlpflichtmodul
  • Lehrsprache: Englisch

Studiengänge, Modulgruppen & Module

Digital Health MA
IT-Systems Engineering MA
  • OSIS: Operating Systems & Information Systems Technology
    • HPI-OSIS-K Konzepte und Methoden
  • OSIS: Operating Systems & Information Systems Technology
    • HPI-OSIS-T Techniken und Werkzeuge
  • OSIS: Operating Systems & Information Systems Technology
    • HPI-OSIS-S Spezialisierung
Data Engineering MA


Attention mechanisms - focusing on the important aspects of data while ignoring less relevant aspects - have been at the forefront of many recent breakthroughs in deep learning. Originally developed for language models, attention models have been adapted to vision, audio & music, graphs, and other data types.

We will read and discuss a number of landmark papers in the area, including recent developments.


Familiarity with deep learning, e.g. from the Deep Learning lecture by Prof. Lippert, or a similar course.

Basic understanding of RNNs such as LSTMs and GRUs, although there will be a short recap in the beginning.

Students are not required to know anything about attention mechanisms before the start of the seminar.


The course is also open for non-HPI students.

Lern- und Lehrformen

This seminar will have a format similar to a reading club/journal club, featuring student presentations and longer discussions. Each week, we will cover one paper related to attention mechanisms in deep learning, starting from the very basics and covering different application areas.


70% of the grade will be based on the paper presentation each student is required to give. Another 30% will be based on active participation in the weekly discussions.


The course starts on April 14th with a kick-off meeting.

Dates: Wednesdays, 13:30 - 15:00 via Zoom.

Zoom link:


Meeting ID: 620 4705 2575
Passcode: 12923653