Hasso-Plattner-Institut
Prof. Dr. Felix Naumann
  
 

Organization

  • Lecture: One lecture per week
  • Exercises: Two practical exercises
  • Audience: Master students
  • Credits: 3 credit points, 2 SWS
  • Room: HS3
  • Date: Wed, 15:15 - 16:45
  • Lecturer: Thorsten Papenbrock

    Related Topics

    Description

    The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization.

    Big Data Analytics is a multi-million dollar market that grows constantly! Data and the ability to control and use it is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements.

    In this lecture, we take a look a various technologies involved in building distributed, data-intensive systems. We discuss theoretical concepts (data models, encoding, replication, ...) as well as some of their practical implementations (Akka, MapReduce, Spark, ...). Since workload distribution is a concept which is useful for many applications, we focus in particular on data analytics.

    Schedule

    Date

    Topic

    18.10.2017 Introduction
    25.10.2017 Foundations
    01.11.2017 Data Models and Query Languages
    08.11.2017 Storage and Retrieval
    15.11.2017 Encoding and Evolution
    22.11.2017 Hands-On: Akka
    29.11.2017 Replication
    06.12.2017 Partitioning
    13.12.2017 Transactions
    20.12.2017 Batch Processing
    03.01.2018 Hands-On: Spark
    10.01.2018 Stream Processing
    17.01.2018 Distributed Systems
    24.01.2018 Consistency and Consensus
    31.01.2018
    07.02.2018

    Course Details

    Prerequisites

    • Interest in data, parallelization, and analytics
    • A little background in database systems
      (e.g. DBS I lecture)
    • Very basic knowledge of the Java programming language
      (because most hands-ons and exercises are in Java)

    Grading

    • A (written) exam determines the final grade
    • Prerequisite to the exam is ...

      • attending the lectures
      • participation in exercises
      • completion of exercise tasks

    Literature

    Course book:

    • Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann, 2017, 978-1449373320

    Further reading:

    • Distributed Systems, Maarten van Steen and Andrew S. Tanenbaum, 2017, 978-1543057386
    • Web-Scale Data Management for the Cloud, Wolfgang Lehner and Kai-Uwe Sattler, 2013, 1489997717
    • Introduction to Parallel Computing, Zbigniew J. Czech, 2017, 978-1107174399
    • Principles of Distributed Database Systems, M. Tamer Özsu and Patrick Valduriez, 2011, 978-1441988331
    • Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services, Brendan Burns, 2017, 978-1491983645
    • Spark: Big Data Cluster Computing in Production, Ilya Ganelin and Ema Orhian and Kai Sasaki and Brennon York, 2016, 978-1119254010
    • Reactive Messaging Patterns with the Actor Model, Vaughn Vernon, 2015, 978-0133846836
    • Mining Massive Datasets, Jure Leskovec and Anand Rajaraman and Jeffrey David Ullman, 2014, 978-1107077232
    • Algorithmische Geometrie, Rolf Klein, 2005, 978-3540209560