Smart metering is the most recent approach to meet the latest challenges in the energy market. Fine-grained information about the current energy consumption is gathered by smart meters and real-time metering. Traditional metering is based on one measurement per year for each customer. In contrast the smart metering approach enables to produce metering data every 15 minutes, which is the default measurement interval of modern metering devices. On one hand, this leverages great opportunities for demand-side management, energy saving and real-time pricing, but on the other hand, an immense amount of data needs to be handled. In Germany, residential customers will produce approximately 1.4 trillion records per year. Ordinary relational databases are overcharged with processing millions of smart meter readings, which tremendously slows down the response time of analyses. This challenge can be managed by using in-memory technology. Each of the bachelor's theses deals with a specific aspect of the overall problem. The image gives an overview of the developed system. Click on a part to view detailed information as presented in the corresponding bachelor's thesis.

• Sten Aechtner: Distributed Database Operations for Energy Data Management
• Oliver Richter: Modeling and Definition of Energy Rates for In-Memory Databases
• Tobias Schubotz: Statistical Operations for the Analysis of Energy Data using In-Memory Databases
• Felix Leupold: Real-Time Pattern Matching in Energy Consumption Data using In-Memory Technology
• Julian Neuhaus: Forecast models and methods in the energy industry
• Steffen Pade: Optimizing Database Queries in Main Memory Databases for the Energy Industry
• Romano Licker: Visualization Methods of Real-Time Energy Data
• Leonhard Schweizer: Optimizing Storage of Energy Event Data in In-Memory Databases

Sten Aechtner: Distributed Database Operations for Energy Data Management

In order to process the huge volume of data it is necessary to partition the data across multiple servers. This is an opportunity but also a challenge. On the one hand partitioning the data across multiple servers introduces more complexity when it comes to accessing the data but on the other hand it enables to process the data in parallel on each machine enhancing the throughput. So partitioning is basically a trade-off between throughput and complexity. This thesis deals with the question how to partition energy data efficiently to both store and analyze the data as fast as possible. The objective is to split the load equally across all nodes and utilize all available resources.

Oliver Richter: Modeling and Definition of Energy Rates for In-Memory Databases

To take advantage of smart metering, energy rates have to be as detailed as the consumption measurements. Therefore a new rate model has been developed which allows pricing any interval of time and thus is able to follow supply and demand of energy. Through appropriate pricing, consumption can be reduced or shifted to balance the taking. Using in-memory technology enables billing quarter-hourly prices for one month of a customer's data in less than three seconds. This includes common cost modifiers like rental fees for metering devices. It is possible to check at anytime what one has to pay, and through the use of prediction even what the costs will be at the end of the month. Rates can be compared based on predicted consumption and exchanged quickly.

Tobias Schubotz: Statistical Operations for the Analysis of Energy Data using In-Memory Databases

Smart metering leverages great opportunities in demand-side management, energy saving and real-time pricing, but to utilize the entire potential, an immense amount of data needs to be handled. In this thesis, use cases and requirements concerning the analysis of energy data are identified. The resulting operations are presented and implemented on an in-memory column-oriented database. Parallelization opportunities as well as real-time capabilities of the considered operations are evaluated. Considered operations have been smoothing algorithms (moving average, running mean), an operation for creating box plots and an operation for dividing energy data into groups of base, medium and peak load times. The implementation was done using SQL functions with embedded R and L code directly on the database. R is an open source language and L is SAP proprietary. These functions where embedded in SQLscript to be able to call a calculation view instead of persisting the function result on the database. For the purpose of comparing the performance, some operations were also implemented in SQL only.

Felix Leupold: Real-Time Pattern Matching in Energy Consumption Data using In-Memory Technology

Today in most manufacturing industries energy costs are the third important cost factor beside material and labor costs. However management and controlling departments suffer from difficulties in identifying the exact energy costs per production step or unit. New technologies in the energy sector, such as smart metering, allow the consumption of different production units to be analyzed at a fine granularity. In a case study, I apply pattern matching algorithms on smart metering data to extract product information from the considered device. I implement the algorithms in a real world application using an in-memory database, which allows analyzing the huge amount of data in real-time on-the-fly. I leverage product identification for short term consumption prediction and fault recognition in the production process. Finally I will discuss in how far our case study can be generalized and used in higher granular architectures.

Julian Neuhaus: Forecast models and methods in the energy industry

In the smart grid of the future it is very important to know anytime, which amount of energy is needed - and that with the best achievable accuracy. Based on the huge amount of historical data it is possible, that every piece of information can be used for the prediction. This allows optimizing the accuracy of a prediction to get the best result. The in-memory technology allows performing real-time predictions in less than one second. Through the capability of parallelization it is also possible to perform every prediction on different CPU cores to speed up the calculation.

Steffen Pade: Optimizing Database Queries in Main Memory Databases for the Energy Industry

Based on the fine-grained information provided by smart metering providers and consumers have diverse possibilities to analyse the behaviour of consumption like the current energy consumption or comparing tariffs based on this. These analyses require huge computing performance due to the large amount of data. While currently analyses are realised by the most modern hardware and data warehouses it also shows that an in-memory database is able to fulfill these requirements and that it provides the capacity for the optimisation of the energy industry’s specific analyses. Optimising these analyses - defined in shape of SQL queries - is the main topic of this thesis. This means evaluating different optimization approaches, examining problems in the query execution and also optimising the underlying data schema. The optimization capabilities of a modern in-memory database are shown considering their influence on the performance of the analyses.

Romano Licker: Visualization Methods of Real-Time Energy Data

Consuming energy is as simple as plugging the plug into a socket. How much energy one actually consumes is displayed on some meter in the basement. With constantly increasing energy prices it becomes more important than ever to make energy consumption data easily accessible. With the recent deployment of smart meters we are now able to get the information from our basement directly to our computer screens as detailed as we want it and in a few years it is most likely that each device will be capable of telling us (or: indicating)how much energy it consumes. But what can be done with this massive amount of data? What is really of interest and what kind of analysis on the data can support the needs of customers? In order to have answers to these questions I created a survey which had 260 participants. Large companies, which consume huge amounts of energy every day, already reduce their costs significantly by monitoring their consumption. I present applications for end-users and enterprises, compare them and highlight visualization solutions.

Leonhard Schweizer: Optimizing Storage of Energy Event Data in In-Memory Databases

This thesis describes an approach to the real-time processing of energy event data. By choosing an in-memory database as storage they can be processed and analyzed simultaneously while notably reducing the amount of required space at the same time through the utilization of compression potentials in column-based tables. As a result, new opportunities arise, like offering electricity rates with real-time pricing or managing supply and demand based on up-to-the-minute analytics.