Autonomous Data Management

The trend of enterprises storing more and more data in cloud-based database systems requires providers to focus on self-optimization and cost-efficiency to stay competitive. We investigate these topics in two connected lines of research.

First, we work on in-memory data management concepts. With our in-memory research database Hyrise, we study how high-performance database systems can optimize themselves with the eventual goal of an autonomous operation without the need for human interaction. To advance in this direction, we are working on various topics, such as data tiering on modern storage technologies, data encoding, index selection, and data dependencies.

The second line of research explores a database system architecture that exploits modern, serverless cloud infrastructure such as Function-as-a-Service (FaaS) platforms to achieve cost-efficiency. With our Skyrise database system, we target the current sweet spot of interactive in-situ analytics on cold data in cloud storage. Skyrise incorporates a number of components from Hyrise and introduces new capabilities to compile and execute queries elastically with cloud functions.

Teaching

We offer lectures, seminars and projects on data management for enterprise applications. For HPI Master's students, we also provide a varity of thesis topics.

Publications

Our research has been published in leading database systems conferences (VLDB, EDBT, ICDE, CIDR, and SIGMOD).

Riekenbrauck, N., Weisgut, M., Lindner, D., Rabl, T.: A Three-Tier Buffer Manager Integrating CXL Device Memory for Database Systems. Joint International Workshop on Big Data Management on Emerging Hardware and Data Management on Virtualized Active Systems @ ICDE 2024 (2024).

[ BibTeX ]

Richly, K., Schlosser, R., Boissier, M.: Budget-Conscious Fine-Grained Configuration Optimization for Spatio-Temporal Applications. Proceedings of the VLDB Endowment. bll. 4079–4092 (2022).

[ BibTeX ] [ Download ]

Justen, D.: Cost-efficiency and Performance Robustness in Serverless Data Exchange. Proceedings of the 2022 International Conference on Management of Data (SIGMOD ’22), Student Research Competition. bll. 2506–2508. Association for Computing Machinery, New York, NY, USA (2022).

[ BibTeX ]

Boissier, M.: Robust and Budget-Constrained Encoding Configurations for In-Memory Database Systems. Proceedings of the VLDB Endowment. bll. 780–793 (2022).

[ BibTeX ] [ Download ]

Bodner, T., Pietz, T., Bollmeier, L.J., Ritter, D.: Doppler: Understanding Serverless Query Execution. Proceedings of the SIGMOD Workshop on Big Data in Emergent Distributed Environments. (2022).

[ BibTeX ]

Kossmann, J., Kastius, A., Schlosser, R.: SWIRL: Selection of Workload-aware Indexes using Reinforcement Learning. 25th International Conference on Extending Database Technology (EDBT 2022). bll. 155–168 (2022).

[ BibTeX ]

Kossmann, J., Lindner, D., Naumann, F., Papenbrock, T.: Workload-driven, Lazy Discovery of Data Dependencies for Query Optimization. Proceedings of the Conference on Innovative Data Systems Research (CIDR) (2022).

[ BibTeX ]

Richly, K., Schlosser, R., Boissier, M.: Joint Index, Sorting, and Compression Optimization for Memory-Efficient Spatio-Temporal Data Management. 37th IEEE International Conference on Data Engineering (ICDE). bll. 1901–1906 (2021).

[ BibTeX ] [ Download ]

Halfpap, S., Schlosser, R.: Memory-Efficient Database Fragment Allocation for Robust Load Balancing when Nodes Fail. 37th IEEE International Conference on Data Engineering, ICDE 2021, Chania, Greece, April 19-22, 2021. bll. 1811–1816 (2021).

[ BibTeX ] [ Download ]

10.

Kossmann, J., Boissier, M., Dubrawski, A., Heseding, F., Mandel, C., Pigorsch, U., Schneider, M., Schniese, T., Sobhani, M., Tsayun, P., Wille, K., Perscheid, M., Uflacker, M., Plattner, H.: A Cockpit for the Development and Evaluation of Autonomous Database Systems. 37th IEEE International Conference on Data Engineering, ICDE. bll. 2685–2688 (2021).

[ BibTeX ] [ Download ]

@inproceedings{kossmann2021cockpit,
  abstract = {Databases are highly optimized complex systems with a multitude of configuration options. Especially in cloud scenarios with thousands of database deployments, determining optimized database configurations in an automated fashion is of increasing importance for database providers. At the same time, due to increased system complexity, it becomes more challenging to identify well-performing configurations. Therefore, research interest in autonomous or self-driving database systems has increased enormously in recent years. Such systems promise both performance improvements and cost reductions. In the literature, various fully or partially autonomous optimization mechanisms exist that optimize single aspects, e.g., index selection. However, database administrators and developers often distrust autonomous approaches, and there is a lack of practical experimentation opportunities that could create a better understanding. Moreover, the interplay of different autonomous mechanisms under complex workloads remains an open question. The presented cockpit enables an interactive assessment of the impact of autonomous components for database systems by comparing (autonomous) systems with different configurations side by side. Thereby, the cockpit enables users to build trust in autonomous solutions by experimenting with such technologies and observing their effects in practice.},
  author = {Kossmann, Jan and Boissier, Martin and Dubrawski, Alexander and Heseding, Fabian and Mandel, Caterina and Pigorsch, Udo and Schneider, Max and Schniese, Til and Sobhani, Mona and Tsayun, Petr and Wille, Katharina and Perscheid, Michael and Uflacker, Matthias and Plattner, Hasso},
  booktitle = {37th IEEE International Conference on Data Engineering, ICDE},
  keywords = {adm autonomous database hyrise in-memory_database mboissierselected myown self-driving self-managing},
  pages = {2685-2688},
  title = {A Cockpit for the Development and Evaluation of Autonomous Database Systems},
  year = 2021
}

11.

Kossmann, J., Papenbrock, T., Naumann, F.: Data dependencies for query optimization: a survey. VLDB Journal. (2021).

[ BibTeX ]

12.

Bodner, T.: Elastic Query Processing on Function as a Service Platforms. Proceedings of the VLDB PhD Workshop. (2020).

[ BibTeX ]

13.

Kossmann, J., Halfpap, S., Jankrift, M., Schlosser, R.: Magic mirror in my hand, which is the best in the land? An Experimental Evaluation of Index Selection Algorithms. Proceedings of the VLDB Endowment. bll. 2382–2395 (2020).

[ BibTeX ] [ Download ]

14.

Dreseler, M., Boissier, M., Rabl, T., Uflacker, M.: Quantifying TPC-H Choke Points and Their Optimizations. Proceedings of the VLDB Endowment. bll. 1206–1220 (2020).

[ BibTeX ] [ Download ]

15.

Halfpap, S.: Efficient Scale-Out Using Query-Driven Workload Distribution and Fragment Allocation. Proceedings of the VLDB 2019 PhD Workshop co-located with the 45th International Conference on Very Large Databases (VLDB 2019) (2019).

[ BibTeX ]

16.

Dreseler, M., Kossmann, J., Boissier, M., Klauck, S., Uflacker, M., Plattner, H.: Hyrise Re-engineered: An Extensible Database System for Research in Relational In-Memory Data Management. 22nd International Conference on Extending Database Technology (EDBT). bll. 313–324 (2019).

[ BibTeX ] [ Download ]

@inproceedings{dreseler2018,
  abstract = {Research in data management profits when the performance evaluation is based not only on individual components in isolation, but uses an actual DBMS end-to-end. Facilitating the integration and benchmarking of new concepts within a DBMS requires a simple setup process, well-documented code, and the possibility to execute both standard and custom benchmarks without tedious preparation. Fulfilling these requirements also makes it easy to reproduce the results later on. The relational open-source database Hyrise (VLDB, 2010) was presented to make the case for hybrid row- and column-format data storage. Since then, it has evolved from being a single- purpose research DBMS towards becoming a platform for various projects, including research in the areas of indexing, data partitioning, and non-volatile memory. With a growing diversity of topics, we have found that the original code base grew to a point where new experimentation became unnecessarily difficult. Over the last two years, we have re-written Hyrise from scratch and built an extensible multi-purpose research DBMS that can serve as an easy-to-extend platform for a variety of experiments and prototyping in database research. In this paper, we discuss how our learnings from the previous version of Hyrise have influenced our re-write. We describe the new architecture of Hyrise and highlight the main components. Afterwards, we show how our extensible plugin architecture facilitates research on diverse DBMS-related aspects without compromising the architectural tidiness of the code. In a first performance evaluation, we show that the execution time of most TPC-H queries is competitive to that of other research databases.},
  author = {Dreseler, Markus and Kossmann, Jan and Boissier, Martin and Klauck, Stefan and Uflacker, Matthias and Plattner, Hasso},
  booktitle = {22nd International Conference on Extending Database Technology (EDBT)},
  keywords = {adm hyrise mboissierselected myown},
  month = 3,
  pages = {313-324},
  title = {Hyrise Re-engineered: An Extensible Database System for Research in Relational In-Memory Data Management},
  year = 2019
}

17.

Kossmann, J.: Self-Driving: From General Purpose to Specialized DBMSs. Proceedings of the VLDB 2018 PhD Workshop co-located with the 44th International Conference on Very Large Databases (VLDB 2018), Rio de Janeiro, Brasil, Aug 27-31, 2018 (2018).

[ BibTeX ]

18.

Boissier, M.: Reducing the Footprint of Main Memory HTAP Systems: Removing, Compressing, Tiering, and Ignoring Data. PhD Workshop at VLDB. CEUR-WS.org (2018).

[ BibTeX ] [ Download ]

Autonomous Data Management

Teaching

Publications

News

22.09.2023 | Trends and Concepts in the Softwareindustry Seminar offered in WiSe 2023/2024

22.05.2023 | Christopher Hagedorn Successfully Defended His PhD Thesis

03.03.2023 | Last Trends and Concepts course of Prof. Hasso Plattner

01.03.2023 | Jan Kossmann Successfully Defended His PhD Thesis

26.02.2023 | Paper on Data Tiering in Hyrise Published in BTW Proceedings

24.02.2023 | Paper on EPIC Research Group Published in SIGMOD Record

30.11.2022 | Paper on Database Optimizations for Spatio-Temporal Data published in PVLDB

04.10.2022 | Günter Hesse Successfully Defended His PhD Thesis

08.07.2022 | Successful PhD Defense by Markus Dreseler

Literature

Contact