Hasso-Plattner-Institut
Prof. Dr. h.c. Hasso Plattner
 

Dynamic Programming and Reinforcement Learning

General Information

  • Teaching staff: Dr. Rainer Schlosser, Alexander Kastius
  • 4 Semesterwochenstunden (SWS) 6 ECTS (graded)
  • Enrollment time: 01 Apr 2021 until 12 Apr 2021
  • Time: Mon 13.30 - 15.00, Thu 11.00 - 12.30
  • Room: online (via Zoom https://zoom.us/j/7271364393 Password: 256757)
  • Maximum Number of Participants: -
  • Specialization areas:
    • IT-Systems Engineering: BPET, OSIS, ITSE-Analyse, ITSE-Entwurf
    • Data Engineering: DATA
  • Documentation deadline: September15, 2021

Short Description

The need for automated decision-making is steadily increasing. Hence, data-driven decision-making techniques are essential. We assume a system that follows certain dynamics and has to be tuned or controlled over time such that certain constraints are satisfied and a specified objective is optimized. Typically, the current state of the system as well as the interplay of rewards and potential future states associated to certain actions have to be taken into account. The dynamics and state transitions may have to be estimated from data using suitable ML-based techniques.

As, in general, exact solution approaches of such dynamic optimization problems do not scale often heuristics have to be used (e.g., in case the number of states is too large, cf. curse of dimensionality). Besides classical approaches such as dynamic programming (DP) state-of-the-art heuristic optimization techniques such as approximate dynamic programming (ADP) or reinforcement learning (RL) are suitable alternatives.

Goals of the Course

Understand...

  • opportunities and challenges of decision-making
  • static deterministic problems
  • stochastic dynamic problems
  • optimization models and solution techniques

Do ...

  • work in small teams
  • set up suitable models, apply optimization techniques 
  • simulate controlled processes, compare performance results

Improve/Learn ...

  • mathematical, analytical, and modelling skills
  • optimization techniques
  • dynamic programming methods
  • reinforcement learning methods

Preconditions

  • the participants should be interested in mathematical methods
  • the number of participants is not restricted

Teaching and Learning Process

The course is a combination of a lecture and a practical part:

  • teachers impart relevant knowledge
  • students work on a self-containing topic in a team of ca. 3 people
  • students present their work

Grading

  • 30% Project Results (projects assigned in June)
  • 70% Project Documentation (deadline Aug 31)

Material/Preparation

Slides and Upcoming Topics

Exercises:

  • Airline Example (Bonus, until April 21), see slides VL1b
  • Inventory Example (until April 28), see slides VL2a

Material: