Prof. Dr. Tilmann Rabl


Prof. Dr. Tilmann Rabl, Nils Straßenburg, Ricardo Salazar


The amount of data that can be generated and stored in academic and industrial projects and applications is increasing rapidly. Big data analytics technologies have established themselves as a solution for big data challenges to the scalability problems of traditional database systems. The vast amounts of new data that is collected, however, usually is not as easily analyzed as curated, structured data in a data warehouse is. Typically, these data are noisy, of varying format and velocity, and need to be analyzed with techniques from statistics and machine learning rather than pure SQL-like aggregations and drill-downs. Moreover, the results of the analyzes frequently are models that are used for decision making and prediction. The complete process of big data analysis is described as a pipeline, which includes data recording, cleaning,

In this lecture, we will discuss big data systems, ie, infrastructures that are used to handle all steps in typical big data processing pipelines.


  • Principles of Distributed Database Systems , M. Tamer Özsu and P. Valduriez, 2011, 978-1441988331
  • Distributed Systems , Maarten van Steen and Andrew S. Tanenbaum, 2017, 978-1543057386
  • Streaming Systems , T. Akidau, S. Chernyak, R. Lax, 2018, 978-1-491-98387-4
  • Designing Data-Intensive Applications , M. Kleppmann, 2017, 978-1449373320


  • The introductory lecture will also be streamed live, Zoom link is available in Moodle
  • Course management will be done using the HPI Moodle
  • The lectures will be held on-site at HPI
  • Non-HPI participants : please send us an email to get access to the Moodle
  • All lectures are recorded and available in Tele-Task

Schedule (tentative) 

The lectures will be held on Tuesdays (L-E.03) and Thursdays (L-E.03) at 11:00 to 12:30 h.

DateTopic TuesdayTopic Thursday
18.10. /20.10Intro / OrganizationalIntro to Programming Exercise I (HS 2)
25.10. / 27.10.Use Case - Search EnginesPerformance Management
1.11. / 3.11.Map Reduce IExercise
8.11. / 10.11.Map Reduce IIMap Reduce III
15.11. / 17.11.Data Center / CloudExercise
22.11 / 24.11.File SystemsKey Value Stores I
29.11. / 1.12.Key Value Stores IIExercise
6.12 / 8.12.Key Value Stores IIIKey Value Stores IV
13.12. / 15.12.Stream Processing IExercise
3.1. / 5.1.Stream Processing IIStream Processing III
10.1./ 12.1.ML Systems IExercise
17.1. / 19.1.ML Systems IIML Systems III
24.1. / 26.1.Modern Hardware IExercise
31.1. / 2.2.Modern Hardware IIIndustry Talk
7.2. / 9.2.Exam PrepExam


The grade will be determined by an exam. The time and location of the exam will be anounced at least 6 weeks in advance. The prerequisite for admission to the exam is the successful completion of the exercises. In case of low participation, the exam might be replaced by an oral examination.

The grade breakdown is as follows:

  • 3 graded programming exercies (50% in each exercise for exam eligibility)
  • 4 graded quizzes (50% in all quizzes for exam eligibility)
  • Final Exam (100% of the grade)