Hasso-Plattner-InstitutSDG am HPI
Hasso-Plattner-InstitutDSG am HPI
  
Login
 

Attention in Deep Learning (Sommersemester 2021)

Lecturer: Prof. Dr. Christoph Lippert (Digital Health - Machine Learning)

General Information

  • Weekly Hours: 2
  • Credits: 4
  • Graded: yes
  • Enrolment Deadline: 18.03.2021 - 09.04.2021
  • Teaching Form: Seminar
  • Enrolment Type: Compulsory Elective Module
  • Course Language: English

Programs & Modules

Digital Health MA
IT-Systems Engineering MA
Data Engineering MA
  • DATA-Konzepte und Methoden
  • DATA-Techniken und Werkzeuge
  • DATA-Spezialisierung

Description

Attention mechanisms - focusing on the important aspects of data while ignoring less relevant aspects - have been at the forefront of many recent breakthroughs in deep learning. Originally developed for language models, attention models have been adapted to vision, audio & music, graphs, and other data types.

We will read and discuss a number of landmark papers in the area, including recent developments.

Requirements

Familiarity with deep learning, e.g. from the Deep Learning lecture by Prof. Lippert, or a similar course.

Basic understanding of RNNs such as LSTMs and GRUs, although there will be a short recap in the beginning.

Students are not required to know anything about attention mechanisms before the start of the seminar.

 

The course is also open for non-HPI students.

Learning

This seminar will have a format similar to a reading club/journal club, featuring student presentations and longer discussions. Each week, we will cover one paper related to attention mechanisms in deep learning, starting from the very basics and covering different application areas.

Examination

70% of the grade will be based on the paper presentation each student is required to give. Another 30% will be based on active participation in the weekly discussions.

Dates

The course starts on April 14th with a kick-off meeting.

Dates: Wednesdays, 13:30 - 15:00 via Zoom.

Zoom link:

uni-potsdam.zoom.us/j/62047052575

Meeting ID: 620 4705 2575
Passcode: 12923653

Zurück