Hasso-Plattner-Institut
Prof. Dr. h.c. Hasso Plattner
  
 

HYRISE - An In-Memory Storage Engine

General Information


Announcement

On October 19, 2017 we released Hyrise2, the new version of Hyrise with a rewritten codebase and new concepts that will support current and future research in in-memory data management. Check out Hyrise2 on GitHub.

Research articles published before October 2017 are based on the previous codebase of Hyrise, which can be found here.


In 2009, our group started to design and develop the in-memory storage engine Hyrise. Our goal is to provide a clean and flexible platform for research in the area of in-memory data management, allowing us, our students and other researchers to conduct experiments around new data management concepts.

Hyrise is completely open source and available on Github: https://github.com/hyrise/hyrise

Project Team: Markus Dreseler, Jan KossmannMartin Boissier, Stefan Klauck, Dr. Matthias Uflacker,  Prof. Dr. h.c. Hasso Plattner

Related Research Area: In-Memory Data Management for Enterprise Systems

Architecture

High-level depiction of Hyrise2's architecture

The figure depicts the high-level architecture of Hyrise. With Hyrise2 we introduce a chunk-based storage concept. Chunks are horizontal partitions of a certain capacity. When the capacity of a chunk is reached, the storage engine will apply dictionary encoding to reduce the memory footprint. The main motivation behind the chunk concept is increased flexibility for data distribution. We envision chunks as a natural entity for distributing data, e.g., in NUMA and replication scenarios, and for offloading data to GPU accelerators. The chunk concept supports different internal data layouts. At the current stage, we have implemented a column store. By implementing different types of chunks, we will also be able to add support for row stores and hybrid data layouts. Hyrise keeps the complete data set in main memory. Persistent storage is utilized to guarantee durability.

About the Rewrite

In 2016, we started to rebuild Hyrise from scratch. While we left most of Hyrise’ proven concepts in place, we wrote an entirely new codebase that will serve as the basis for our future research projects. This effort was driven by a number of both functional and nonfunctional requirements:

  • The original code was developed at a time when modern C++ features like smart pointers and move semantics were not common. Integrating these features into an existing codebase is cumbersome and prone to errors. With the new version of Hyrise, we introduced C++17 features in order to improve readability, maintainability, and performance of the database.
  • Rewriting core data structures such as tables and internal data stores gave us the possibility to apply some of the learnings of the past years. We found that the high number of virtual method calls in the previous version of Hyrise caused a significant overhead during execution. This number has been considerably reduced in Hyrise2.
  • The design of Hyrise2 much better accounts for NUMA awareness and NUMA-related optimizations.
  • In the previous version of Hyrise, query plans had to be written by hand, creating and connecting physical operators using our own JSON format. This was a tiresome and error-prone process that resulted in long files that were hard to maintain.
  • With the new version of Hyrise, we have built a full SQL pipeline that starts with our own SQL Parser, translates the SQL query into an abstract syntax tree (AST), performs a number of rule-based optimizations, and returns an executable query plan.
  • For better reproducibility of our research results and to allow for the use of Hyrise in practical exercises of our database lectures, we established a simple setup process. Using the install script and explicit dependency management, Hyrise2 can be set up in three steps and in under ten minutes.

The following sections give an overview of the research areas we are currently addressing with Hyrise2.

Footprint Reduction

Contact: Martin Boissier 

Hyrise2 in its vanilla version is a fully DRAM-resident database. But with ever growing data volumes being processed by modern applications, DRAM contingents of single servers (and even large clusters) are easily exceeded. Not even mentioning hardware costs of storing all data in comparatively expensive DRAM. The focus point of Footprint Reduction studies topics that aim to reduce the DRAM space used by Hyrise2.

While other storage tiers besides DRAM are less expensive, they also introduce performance penalties. Consequently, the question is to which extend we can lower the DRAM footprint without loosing the performance superiority of DRAM-resident databases. And with that: how can we balance this trade-off?

Our research focusses on two areas that we work on separately in order to limit dependencies and the database's complexity:

  • Compression: part of this area is the challenge of reducing the required DRAM space for data structures, e.g., via lossless compression or approximate indexing structures.
  • Tiering: the goal of data tiering is to (i) classify data as 'tierable' (usually infrequently accessed data) and (ii) find means to efficiently evict this data to secondary storage layers. Furthermore, we study means to avoid unnecessary accesses to secondary storage as often as possible.

We are currently working on several projects that aim to improve the footprint. Amongst them is workload-driven and adaptable column compression scheme, approximate indexing structures for faster access paths and improved cardinality estimations, and filters to improve partition pruning for tiered data.

New Memory Hardware

Contact: Markus Dreseler

Memory is the new bottleneck of modern databases. In the time that it takes to retrieve a single memory address from DRAM, hundreds of other non-memory operations can be computed. As such, it becomes increasingly important to make efficient use of the available memory bandwidth. When looking at large NUMA setups where latencies are higher and bandwidths lower, we are exploring ways to make direct use of hardware primitives that allow us to improve the access performance.

Looking at upcoming memory technologies, such as Non-Volatile Memory (NVM), we are implementing concepts that incorporate these into the memory management of the database. We identified two research areas: First, a database that uses NVM as its primary storage can guarantee crash resilience even without a separate log component. When persistence becomes a first-class citizen, this allows us to rethink the way transaction handling and concurrency work. Furthermore, native persistence without the need of replaying a log greatly reduces the time needed to restart and recover the database.

Second, we are looking into how NVM and DRAM can be used side-by-side. In a system that has both types of memory, data has to be dynamically placed so that the advantages and disadvantages of both types are carefully balanced. Some data structure, for example temporary hash tables created by joins are a natural fit for fast, volatile DRAM. Others, such as persistent attribute vectors that are mostly read sequentially, should be placed on NVM instead.

Replication

Contact: Stefan Klauck

Increasing demand for analytical processing capabilities can be managed by scale-out approaches. In replication schemes, replica nodes process read-only queries on snapshots of the master node without violating transactional consistency. When analyzing workloads, we can identify query access patterns and replicate data depending to its access frequency. In this way, we can reduce the replication cluster’s memory consumption and process the query load at lower cost.

In our research, we investigate approaches to find good replication configurations that can be deployed to scale the workload and reduce the overall storage footprint. Further we compare synchronization methods to keep the replica’s data up-to-date with regards to the transactional changes.

Self-Adapting Databases

Contact: Jan Koßmann

The settings and physical layout of database systems that handle large enterprise workloads need to be configured in order to deliver optimal performance, comply with SLAs, and utilize the underlying hardware’s capabilities to their full extent. These configurations are usually conducted manually by database administrators (DBA). A variety of aspects let manual tuning and configuration appear especially complicated, time-consuming and therefore, expensive.

In realistic databases, there is a large amount of different options to choose from: databases can contain hundreds or thousands of tables. Often, there is a certain interdependence between these options, meaning that they influence each other. In addition, there is no configuration that performs best for all workloads. Every database instance has to be configured individually. Even more, workloads may change over time which makes reconfigurations necessary.

Therefore, we define a set of cost functions and formulate the configuration questions as optimization problems. With data of the database’s internal statistic component, the query plan cache and some parts of the optimizer we investigate how to apply heuristics and solvers that identify optimized solutions for these questions. In the end, these insights can support database administrators or lead to autonomous databases that configure and them themselves. 

General-Purpose GPUs

Contact: Christopher Schmidt

Graphics Processing Units (GPU) are well suited to processing large amounts of data in parallel. With their large number of processing cores and multiple arithmetic logic units (ALUs), GPUs deliver high computational power and better energy efficiency compared to CPUs. In addition, modern GPUs incorporate high bandwidth memory (HBM), which provides a memory bandwidth that exceeds that of DRAM.

We have identified the following key challenges when it comes to incorporating GPUs into in-memory database management systems for better performance:

  • Limited local memory: the available HBM on a GPU is still relatively limited in size (16 GB for the NVIDIA P100)
  • Transfer costs: Data residing in DRAM is required to be transferred to the GPU’s HBM, with costs that potentially nullify the benefits of parallel processing on the GPU
  • Workload selection: GPUs perform well on large amounts of parallel arithmetic operations. For sequential processing and operations beyond arithmetics, the device cannot reach its peak performance and falls behind CPUs.

Upcoming GPU generations allow us to better cope with challenges. For example, Nvidia is providing a high bandwidth interconnect between CPUs and GPUs called NVLink. NVLink can relax the bandwidth limitation between DRAM and on-chip HBM and is used to create clusters of multiple GPUs operating together. This allows for systems of GPUs with up to 128 GB of on-chip memory (Nvidia DGX-1).

For Hyrise we envision and plan to evaluate a setup in which the compressed chunks of every table are completely moved to GPU memory. All inserts are still handled by the CPU and stored in uncompressed chunks in DRAM. Once a modifiable chunk reaches its maximum capacity, it will be compressed and moved to GPU memory. This is a one-time transfer effort as data residing in compressed chunks is not subject to updates (insert-only).

Selected Publications

  • Boissier, M., Spivak, A., Meyer, C.: Improving Materialization for Tiered Column-Stores: A Workload-Aware Ansatz Based on Table Reordering. ACSW '17 Proceedings of the Australasian Computer Science Week Multiconference, ACSW '17. bll. 25:1-25:10. ACM, New York, NY, USA (2017).
     
  • Schwalb, D., Bk, G.K., Dreseler, M., S, A., Faust, M., Hohl, A., Berning, T., Makkar, G., Plattner, H., Deshmukh, P.: Hyrise-NV: Instant Recovery for In-Memory Databases using Non-Volatile Memory. International Conference on Database Systems for Advanced Applications (DASFAA) (2016).
     
  • Schwalb, D., Faust, M., Dreseler, M., Flemming, P., Plattner, H.: Hyrise-NV: Leveraging Non-Volatile Memory for Instant Restarts of In-Memory Database Systems. International Conference on Data Engineering (ICDE) (2016).
     
  • Schwalb, D., Dreseler, M., Uflacker, M., Plattner, H.: NVC-Hashmap: A Persistent and Concurrent Hashmap For Non-Volatile Memories. In-Memory Data Management Workshop (IMDM), in conjunction with VLDB (2015).
     
  • Schwalb, D., Kossmann, J., Faust, M., Klauck, S., Uflacker, M., Plattner, H.: Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications. Proceedings of the 3rd VLDB Workshop on In-Memory Data Mangement and Analytics (IMDM), in conjunction with VLDB 2015 Kohala Coast, Hawaii (2015).
     
  • Meyer, C., Boissier, M., Michaud, A., Vollmer, J.O., Taylor, K., Schwalb, D., Uflacker, M., Roedszus, K.: Dynamic and Transparent Data Tiering for In-Memory Databases in Mixed Workload Environments. International Workshop on Accelerating Data Management Systems Using Modern Processor and Storage Architectures - ADMS @ VLDB 2015 (2015).
     
  • Faust, M., Schwalb, D., Plattner, H.: Composite Group-Keys: Space-efficient Indexing of Multiple Columns for Compressed In-Memory Column Stores. IMDM in conjunction with VLDB (2014).
     
  • Schwalb, D., Faust, M., Wust, J., Grund, M., Plattner, H.: Efficient Transaction Processing for Hyrise in Mixed Workload Environments. IMDM in conjunction with VLDB (2014).
     
  • Wust, J., Grund, M., Plattner, H.: TAMEX: A Task-Based Query Execution Framework for Mixed Enterprise Workloads on In-Memory Databases. IMDM, INFORMATIK 2013 (2013).
     
  • Schwalb, D., Faust, M., Krüger, J.: A Workload-Aware Merge Process for In-Memory Column Stores. CICCI (2013).
     
  • Grund, M., Cudre-Mauroux, P., Krüger, J., Madden, S., Plattner, H.: An overview of HYRISE - a Main Memory Hybrid Storage Engine. IEEE Data Engineering Bulletin. (2012).
     
  • Faust, M., Krüger, J., Schwalb, D., Plattner, H.: Fast Lookups for In-Memory Column Stores: Group-Key Indices, Lookup and Maintenance. ADMS (in conjunction with VLDB) (2012).
     
  • Grund, M., Krüger, J., Plattner, H., Zeier, A., Cudre-Mauroux, P., Madden, S.: HYRISE - A Hybrid Main Memory Storage Engine. Proceedings of the VLDB Endowment Volume 4 Issue 2. bll. 105-116 (2011).
     
  • Grund, M., Cudre-Mauroux, P., Madden, S.: A Demonstration of HYRISE- A Main Memory Hybrid Storage Engine. VLDB (2011).