Prof. Dr. Tilmann Rabl

Prof. Matthias Boehm

Affiliation: TU Graz
Title: DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines
Slides: PDF



Integrated data analysis (IDA) pipelines -- that combine data management (DM) and query processing, high-performance computing (HPC), and machine learning (ML) training and scoring -- become increasingly common in practice. Interestingly, systems of these areas share many compilation and runtime techniques, and the used—increasingly heterogeneous—hardware infrastructure converges as well. Yet, the programming paradigms, cluster resource management, data formats and representations, as well as execution strategies differ substantially. DAPHNE is an open and extensible system infrastructure for such IDA pipelines, including language abstractions, compilation and runtime techniques, multi-level scheduling, hardware (HW) accelerators, and computational storage for increasing productivity and eliminating unnecessary overheads. In this talk, we make a case for IDA pipelines, describe the overall DAPHNE system architecture, its key components, and the design of a vectorized execution engine for computational storage, HW accelerators, as well as local and distributed operations. We further report on the progress of building the DAPHNE compiler and runtime prototype, and the plans on establishing a striving open source community around DAPHNE.

Short CV

Matthias Boehm is a BMK-endowed professor for data management at Graz University of Technology, Austria, and a research area manager for data management at the co-located Know-Center GmbH. His cross-organizational research group focuses on high-level, data science-centric abstractions as well as systems and tools to execute these tasks in an efficient and scalable manner. Prior to joining TU Graz in 2018, he was a research staff member at IBM Research - Almaden, CA, USA, with a major focus on compilation and runtime techniques for declarative, large-scale machine learning in Apache SystemML. Matthias received his Ph.D. from Dresden University of Technology, Germany in 2011 with a dissertation on cost-based optimization of integration flows. His previous research also includes systems support for time series forecasting as well as in-memory indexing and query processing. Matthias is a recipient of the 2016 VLDB Best Paper Award, a 2016 SIGMOD Research Highlight Award, a 2016 IBM Pat Goldberg Memorial Best Paper Award, and the 2021 SIGMOD DS&E Best Paper Award.