The mixed database workloads of enterprise applications are comprised of short-running transactional as well as analytical queries with resource-intensive data aggregations. In this context, caching the query results of long-running aggregate queries is desirable as it increases the overall performance. In-memory databases with a main-delta architecture are optimized for a new caching mechanism for aggregate queries which is the main contribution of this ongoing research project. With the separation into main and delta storage, cached aggregates do not have to be invalidated when new data is inserted to the delta storage. Instead, we can use the cached aggregate query result and combine it with the newly added records in the delta storage. The aggregate cache can be leveraged for a variety of enterprise applications such as financial accounting or simulation scenarios. It does not only speed-up recurring aggregate queries by orders of magnitudes, but also reduces the load on the database as one means to scale out.

For the validation of the aggregates caching concept, we built a prototype that combines both, financial accounting and managerial accounting. We designed a data schema that is based on a unified journal entry line item structure, incorporating fields from financial accounting (BKPF and BSEG) as well as from managerial accounting (COBK, COEP, CE4xxxx, CO-OM, CO-PA). The data consists of 330M records and has been taken from a live system covering 4 years of customer data. We selected use cases together with the customer, adapted actual data to fit with new data model, created fitting plan data, built reports according to the customer's mainly used reports (workload), and measured the resulting performance.

The financial accounting application that demonstrates the benefit of the implemented caching mechanism is a HTML5 application that runs directly on a columnar in-memory database. The caching logic is implemented within the database and is transparent to the application. To visualize the inner mechanics of our caching mechanism, we have built a separate application that displays the performance characteristics of our application. It monitors the execution time of the database queries and the CPU load of the database server. With the possibility to simulate an increased database load on the server, we can show that the execution times increase. When enabling the cache, these execution times drop to nearly zero for repeating queries despite the high load. When we start to insert records to the database, the caching mechanism considers these for the calculation of the final result while the query execution time does not increase significantly.

Project Team: Prof. Dr. h.c. Hasso Plattner, Stephan Müller, Lars Butzmann, Stefan Klauck

Related Research Area: In-Memory Enterprise Data Management

Project Period: since Jun 2011

Related Publications

1. Stephan Müller, Lars Butzmann, Kai Höwelmeyer, Stefan Klauck, Hasso Plattner: Efficient View Maintenance for Enterprise Applications in Columnar In-Memory Databases, 17th IEEE International Enterprise Distributed Object Computing Conference (EDOC), Vancouver, Canada, 2013 
2. Stephan Müller, Hasso Plattner: Aggregates Caching in Columnar In-Memory Databases, 1st International Workshop on In-Memory Data Management and Analytics (IMDM), in conjunction with VLDB 2013, Riva del Garda, Trento, Italy, 2013 
3. Stephan Müller, Lars Butzmann, Stefan Klauck, Hasso Plattner: Workload-Aware Aggregate Maintenance in Columnar In-Memory Databases, IEEE International Conference on Big Data (IEEE Big Data 2013), Silicon Valley, USA, 2013 
4. Stephan Müller, Hasso Plattner: Aggregates Caching for Enterprise Applications, 30th International Conference on Data Engineering (ICDE), PHD Symposium, Chicago, USA, 2014 
5. Stephan Müller, Ralf Diestelkämper, Hasso Plattner: Cache Management for Aggregates in Columnar In-Memory Databases, The Sixth International Conference on Advances in Databases, Knowledge, and Data Applications (DBKDA), Chamonix, France, 2014 
6. Stephan Müller, Lars Butzmann, Hasso Plattner: Efficient Aggregate Cache Revalidation in an In-Memory Column Store,The Sixth International Conference on Advances in Databases, Knowledge, and Data Applications (DBKDA), Chamonix, France, 2014