Hasso-Plattner-InstitutSDG am HPI
Hasso-Plattner-InstitutDSG am HPI
  
Login
 

Attention in Deep Learning (Sommersemester 2021)

Dozent: Prof. Dr. Christoph Lippert (Digital Health - Machine Learning)

Allgemeine Information

  • Semesterwochenstunden: 2
  • ECTS: 4
  • Benotet: Ja
  • Einschreibefrist: 18.03.2021 - 09.04.2021
  • Lehrform: Seminar
  • Belegungsart: Wahlpflichtmodul
  • Lehrsprache: Englisch

Studiengänge & Module

Digital Health MA
  • SCAD-Concepts and Methods
  • SCAD-Technologies and Tools
  • SCAD-Specialization
  • APAD-Concepts and Methods
  • APAD-Technologies and Tools
  • APAD-Specialization
IT-Systems Engineering MA
Data Engineering MA

Beschreibung

Attention mechanisms - focusing on the important aspects of data while ignoring less relevant aspects - have been at the forefront of many recent breakthroughs in deep learning. Originally developed for language models, attention models have been adapted to vision, audio & music, graphs, and other data types.

We will read and discuss a number of landmark papers in the area, including recent developments.

Voraussetzungen

Familiarity with deep learning, e.g. from the Deep Learning lecture by Prof. Lippert, or a similar course.

Basic understanding of RNNs such as LSTMs and GRUs, although there will be a short recap in the beginning.

Students are not required to know anything about attention mechanisms before the start of the seminar.

 

The course is also open for non-HPI students.

Lern- und Lehrformen

This seminar will have a format similar to a reading club/journal club, featuring student presentations and longer discussions. Each week, we will cover one paper related to attention mechanisms in deep learning, starting from the very basics and covering different application areas.

Leistungserfassung

70% of the grade will be based on the paper presentation each student is required to give. Another 30% will be based on active participation in the weekly discussions.

Termine

The course starts on April 14th with a kick-off meeting.

Dates: Wednesdays, 13:30 - 15:00 via Zoom.

Zoom link:

uni-potsdam.zoom.us/j/62047052575

Meeting ID: 620 4705 2575
Passcode: 12923653

Zurück