Prof. Dr. Tilmann Rabl


Prof. Dr. Tilmann Rabl, Ilin Tolovski, Thomas Bodner


The amount of data that can be generated and stored in academic and industrial projects and applications is increasing rapidly. Big data analytics technologies have established themselves as a solution for big data challenges to the scalability problems of traditional database systems. The vast amounts of new data that is collected, however, usually is not as easily analyzed as curated, structured data in a data warehouse is. Typically, these data are noisy, of varying format and velocity, and need to be analyzed with techniques from statistics and machine learning rather than pure SQL-like aggregations and drill-downs. Moreover, the results of the analyses frequently are models that are used for decision making and prediction. The complete process of big data analysis is described as a pipeline, which includes data recording, cleaning, integration, modeling, and interpretation.

In this lab course, we will gain practical experience with big data systems. We will setup a small cluster of Raspberry Pis, install all required software, and implement and execute typical big data tasks.


Successful completion or current participation in the course Big Data Systems.


  • The introductory lab session will also be streamed live, Zoom link is available in Moodle
  • Course management will be done using the HPI Moodle
  • The lab sessions will be held on-site at HPI
  • Non-HPI participants: please send us an email to get access to the Moodle

Schedule (tentative)

The lab sessions will be held on Tuesdays (F-E.06) at 13:30 to 15:00 h.

24.10.Cluster SetupLecture
7.11.Cluster SetupLab
14.11.Spark & Spark SQLLecture
21.11.Spark & Spark SQLLab
28.11.PySpark & Data VisualizationLecture
5.12.PySpark & Data VisualizationLab
12.12.Key Value StoresLecture
19.12.Key Value StoresLab
9.1.Stream ProcessingLecture
16.1.Stream ProcessingLab
23.1.Deep Learning Model TrainingLecture     
30.1.Deep Learning Model TrainingLab


The grade will be determined by an exam. The time and location of the exam will be anounced at least 6 weeks in advance. The prerequisite for admission to the exam is the successful completion of the exercises. In case of low participation, the exam might be replaced by an oral examination.

The grade breakdown is as follows:

  • Group assignments submitted weekly (pass all for exam eligibility)
  • One group presents their solution in the lab sessions
  • Final Exam (100% of the grade)