Guenter Hesse

Research Assistant, PhD Candidate

	Phone	+49 (331) 5509 - 1381
	Fax	+49 (331) 97992 - 579
	E-Mail	guenter.hesse(at)hpi.de
	Room	V-2.02 (Campus II)
	Profiles	LinkedIn, XING, ResearchGate, Google Scholar

Research

Benchmarking Enterprise Data Stream Processing Architectures

Stream processing systems have gained popularity due to developments such as Internet of Things or Industry 4.0. Enterprises benefit from this technology by augmenting their business data with up-to-date streaming information in a stream processing architecture. The performance characteristics of such an architecture, e.g., latency or throughput, vary between data stream processing systems, system configurations, hardware setups, or further environmental differences. A common way of analyzing and evaluating a system's performance is benchmarking. Despite the increasing variety of stream processing frameworks, there is a lack of a satisfying benchmark. This is especially true for the enterprise context, i.e., when combining business or historical data with streaming data. My research investigates how a performance benchmark for streaming architectures should be designed and implemented.

Particularly, it presents a novel benchmark for enterprise streaming architectures. This includes the design of a benchmark process that ensures objective comparisons. As part of the benchmark, we developed a comprehensive toolkit with a focus on usability, which supports the benchmark execution and calculates benchmark results independently of the used stream processing system. The process design also comprises the definition of real-world benchmark queries that are validated in industry and which need to be implemented on the system under test. These queries cover the core functionalities of stream processing systems that are defined in literature. To validate the proposed benchmark, we will analyze selected systems, Apache Flink and Apache Spark Streaming being two of them.

Keywords: Data Stream Processing, Performance Benchmarking, Industry 4.0, and Internet of Things

Teaching

Winter Term 2020/2021

Bachelor's Project: Reinvent the Development Experience for the Intelligent Enterprise

Winter Term 2019/2020

Trends and Concepts in Software Industry II

Summer Term 2019

Trends and Concepts in Software Industry I

Winter Term 2018/2019

Trends and Concepts in Software Industry II

Summer Term 2018

Trends and Concepts in Software Industry I

Application of Causal Inference in Automotive Production

Winter Term 2017/2018

Application of Causal Inference in Automotive Production

Summer Term 2017

Trends and Concepts in Software Industry I

Summer Term 2016

Unternehmensanwendungen: Prozesse, Modelle und Implementierung

Developing Applications for the Internet of Things

Winter Term 2015/2016

Developing Applications for the Internet of Things

Program Committee Membership

IEEE BigData 2020

Publications

Hesse, G., Matthies, C., Perscheid, M., Uflacker, M., Plattner, H.: ESPBench: The Enterprise Stream Processing Benchmark.ACM/SPEC Internation Conference on Performance Engineering (ICPE). pp. 201–212 (2021).
[ BibTeX ] [ Download ]

@inproceedings{hesse2021espbench, abstract = {Growing data volumes and velocities in fields such as Industry 4.0 or the Internet of Things have led to the increased popularity of data stream processing systems. Enterprises can leverage these developments by enriching their core business data and analyses with up-to-date streaming data. Comparing streaming architectures for these complex use cases is challenging, as existing benchmarks do not cover them. ESPBench is a new enterprise stream processing benchmark that fills this gap. We present its architecture, the benchmarking process, and the query workload. We employ ESPBench on three state-of-the-art stream processing systems, Apache Spark, Apache Flink, and Hazelcast Jet, using provided query implementations developed with Apache Beam. Our results highlight the need for the provided ESPBench toolkit that supports benchmark execution, as it enables query result validation and objective latency measures.}, author = {Hesse, Guenter and Matthies, Christoph and Perscheid, Michael and Uflacker, Matthias and Plattner, Hasso}, booktitle = {ACM/SPEC Internation Conference on Performance Engineering (ICPE)}, keywords = {imported esb benchmarking}, pages = {201–212}, title = {ESPBench: The Enterprise Stream Processing Benchmark}, year = 2021 }
Hesse, G., Matthies, C., Uflacker, M.: How Fast Can We Insert? An Empirical Performance Evaluation of Apache Kafka.IEEE International Conference on Parallel and Distributed Systems (ICPADS). pp. 641-648 (2021).
[ BibTeX ] [ Download ]

@inproceedings{hesse2021insert, abstract = {Message brokers see widespread adoption in modern IT landscapes, with Apache Kafka being one of the most employed platforms. These systems feature well-defined APIs for use and configuration and present flexible solutions for various data storage scenarios. Their ability to scale horizontally enables users to adapt to growing data volumes and changing environments. However, one of the main challenges concerning message brokers is the danger of them becoming a bottleneck within an IT architecture. To prevent this, knowledge about the amount of data a message broker using a specific configuration can handle needs to be available. In this paper, we propose a monitoring architecture for message brokers and similar Java Virtual Machine-based systems. We present a comprehensive performance analysis of the popular Apache Kafka platform using our approach. As part of the benchmark, we study selected data ingestion scenarios with respect to their maximum data ingestion rates. The results show that we can achieve an ingestion rate of about 420,000 messages/second on the used commodity hardware and with the developed data sender tool.}, author = {Hesse, Guenter and Matthies, Christoph and Uflacker, Matthias}, booktitle = {IEEE International Conference on Parallel and Distributed Systems (ICPADS)}, keywords = {imported esb kafka}, pages = {641-648}, title = {How Fast Can We Insert? An Empirical Performance Evaluation of Apache Kafka}, year = 2021 }
Matthies, C., Dobrigkeit, F., Hesse, G.: Mining for Process Improvements: Analyzing Software Repositories in Agile Retrospectives.IEEE/ACM International Conference on Software Engineering Workshops (ICSEW). pp. 189-190. ACM, New York, NY, USA (2020).
[ BibTeX ] [ Download ]

@inproceedings{matthies2020mining, abstract = {Software Repositories contain knowledge on how software engineering teams work, communicate, and collaborate. It can be used to develop a data-informed view of a team's development process, which in turn can be employed for process improvement initiatives. In modern, Agile development methods, process improvement takes place in Retrospective meetings, in which the last development iteration is discussed. However, previously proposed activities that take place in these meetings often do not rely on project data, instead depending solely on the perceptions of team members. We propose new Retrospective activities, based on mining the software repositories of individual teams, to complement existing approaches with more objective, data-informed process views.}, address = {New York, NY, USA}, author = {Matthies, Christoph and Dobrigkeit, Franziska and Hesse, Guenter}, booktitle = {IEEE/ACM International Conference on Software Engineering Workshops (ICSEW)}, keywords = {software_engineering agile}, pages = {189-190}, publisher = {ACM}, series = {ICSEW'20}, title = {Mining for Process Improvements: Analyzing Software Repositories in Agile Retrospectives}, year = 2020 }
Matthies, C., Hesse, G.: Towards Using Data to Inform Decisions in Agile Software Development: Views of Available Data.International Conference on Software Technologies (ICSOFT). pp. 552-559. SciTePress (2019).
[ BibTeX ] [ Download ]

@inproceedings{matthies2019ICSOFT, abstract = {Software development comprises complex tasks which are performed by humans. It involves problem solving, domain understanding and communication skills as well as knowledge of a broad variety of technologies, architectures, and solution approaches. As such, software development projects include many situations where crucial decisions must be made. Making the appropriate organizational or technical choices for a given software team building a product can make the difference between project success or failure. Software development methods have introduced frameworks and sets of best practices for certain contexts, providing practitioners with established guidelines for these important choices. Current Agile methods employed in modern software development have highlighted the importance of the human factors in software development. These methods rely on short feedback loops and the self-organization of teams to enable collaborative decision making. While Agile methods stress the importance of empirical process control, i.e. relying on data to make decisions, they do not prescribe in detail how this goal should be achieved. In this paper, we describe the types and abstraction levels of data and decisions within modern software development teams and identify the benefits that usage of this data enables. We argue that the principles of data-driven decision making are highly applicable, yet underused, in modern Agile software development.}, author = {Matthies, Christoph and Hesse, Guenter}, booktitle = {International Conference on Software Technologies (ICSOFT)}, keywords = {software_engineering agile}, organization = {INSTICC}, pages = {552-559}, publisher = {SciTePress}, title = {Towards Using Data to Inform Decisions in Agile Software Development: Views of Available Data}, year = 2019 }
Hesse, G., Sinzig, W., Matthies, C., Uflacker, M.: Application of Data Stream Processing Technologies in Industry 4.0: What is Missing?International Conference on Data Science, Technology and Applications (DATA). pp. 304-310. SciTePress (2019).
[ BibTeX ] [ Download ]

@inproceedings{hesse2019application, abstract = {Industry 4.0 is becoming more and more important for manufacturers as the developments in the area of Internet of Things advance. Another technology gaining more attention is data stream processing systems. Although such streaming frameworks seem to be a natural fit for Industry 4.0 scenarios, their application in this context is still low. The contributions in this paper are threefold. Firstly, we present industry findings that we derived from site inspections with a focus on Industry 4.0. Moreover, our view on Industry 4.0 and important related aspects is elaborated. As a third contribution, we illustrate our opinion on why data stream processing technologies could act as an enabler for Industry 4.0 and point out possible obstacles on this way.}, author = {Hesse, Guenter and Sinzig, Werner and Matthies, Christoph and Uflacker, Matthias}, booktitle = {International Conference on Data Science, Technology and Applications (DATA)}, keywords = {esb industry4.0 data_stream_processing}, organization = {INSTICC}, pages = {304-310}, publisher = {SciTePress}, title = {Application of Data Stream Processing Technologies in Industry 4.0: What is Missing?}, year = 2019 }
Hesse, G., Matthies, C., Glass, K., Huegle, J., Uflacker, M.: Quantitative Impact Evaluation of an Abstraction Layer for Data Stream Processing Systems.IEEE International Conference on Distributed Computing Systems (ICDCS). pp. 1381-1392 (2019).
[ BibTeX ] [ Download ]

@inproceedings{hesse2019quantitative, abstract = {With the demand to process ever-growing data volumes, a variety of new data stream processing frameworks have been developed. Moving an implementation from one such system to another, e.g., for performance reasons, requires adapting existing applications to new interfaces. Apache Beam addresses these high substitution costs by providing an abstraction layer that enables executing programs on any of the supported streaming frameworks. In this paper, we present a novel benchmark architecture for comparing the performance impact of using Apache Beam on three streaming frameworks: Apache Spark Streaming, Apache Flink, and Apache Apex. We find significant performance penalties when using Apache Beam for application development in the surveyed systems. Overall, usage of Apache Beam for the examined streaming applications caused a high variance of query execution times with a slowdown of up to a factor of 58 compared to queries developed without the abstraction layer. All developed benchmark artifacts are publicly available to ensure reproducible results.}, author = {Hesse, Guenter and Matthies, Christoph and Glass, Kelvin and Huegle, Johannes and Uflacker, Matthias}, booktitle = {IEEE International Conference on Distributed Computing Systems (ICDCS)}, keywords = {imported esb data_stream_processing}, pages = {1381-1392}, title = {Quantitative Impact Evaluation of an Abstraction Layer for Data Stream Processing Systems}, year = 2019 }
Hesse, G., Matthies, C., Sinzig, W., Uflacker, M.: Adding Value by Combining Business and Sensor Data: An Industry 4.0 Use Case. In: Li, G., Yang, J., Gama, J., Natwichai, J., and Tong, Y. (eds.) International Conference on Database Systems for Advanced Applications (DASFAA). pp. 528-532. Springer International Publishing (2019).
[ BibTeX ] [ Download ]

@inproceedings{hesse2019adding, abstract = {Industry 4.0 and the Internet of Things are recent developments that have lead to the creation of new kinds of manufacturing data. Linking this new kind of sensor data to traditional business information is crucial for enterprises to take advantage of the data's full potential. In this paper, we present a demo which allows experiencing this data integration, both vertically between technical and business contexts and horizontally along the value chain. The tool simulates a manufacturing company, continuously producing both business and sensor data, and supports issuing ad-hoc queries that answer specific questions related to the business. In order to adapt to different environments, users can configure sensor characteristics to their needs.}, author = {Hesse, Guenter and Matthies, Christoph and Sinzig, Werner and Uflacker, Matthias}, booktitle = {International Conference on Database Systems for Advanced Applications (DASFAA)}, editor = {Li, Guoliang and Yang, Jun and Gama, Joao and Natwichai, Juggapong and Tong, Yongxin}, keywords = {esb sensor_data data_integration industry4.0 business_data}, month = {April}, pages = {528-532}, publisher = {Springer International Publishing}, title = {Adding Value by Combining Business and Sensor Data: An Industry 4.0 Use Case}, volume = 11448, year = 2019 }
Matthies, C., Dobrigkeit, F., Hesse, G.: An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives.IEEE/ACM International Workshop on Bots in Software Engineering (BotSE). pp. 34-37. IEEE (2019).
[ BibTeX ] [ Download ]

@inproceedings{matthies2019BotSE, abstract = {With the recent advances in natural-language processing, combined with the ability to analyze large amounts of data across various domains, software bots can become virtual team members, providing an additional set of automated eyes and additional perspectives for informing and supporting teamwork. In this paper, we propose employing chatbots in the domain of software development with a focus on supporting analyses and measurements of teams' project data. The software project artifacts produced by agile teams during regular development activities, e.g. commits in a version control system, represent detailed information on how a team works and collaborates. Analyses of this data are especially relevant for agile retrospective meetings, where adaptations and improvements to the executed development process are discussed. Development teams can use these measurements to track the progress of identified improvement actions over development iterations. Chatbots provide a convenient user interface for interacting with the outcomes of retrospectives and the associated measurements in a chat-based channel that is already being employed by team members.}, author = {Matthies, Christoph and Dobrigkeit, Franziska and Hesse, Guenter}, booktitle = {IEEE/ACM International Workshop on Bots in Software Engineering (BotSE)}, keywords = {software_engineering}, month = {may}, pages = {34-37}, publisher = {IEEE}, title = {An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives}, year = 2019 }
Matthies, C., Teusner, R., Hesse, G.: Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts.IEEE Frontiers in Education Conference (FIE). pp. 1-9 (2018).
[ BibTeX ] [ Download ]

@inproceedings{matthies2018fie, abstract = {This Innovative Practice Full Paper presents an approach of using software development artifacts to gauge student behavior and the effectiveness of changes to curriculum design. There is an ongoing need to adapt university courses to changing requirements and shifts in industry. As an educator it is therefore vital to have access to methods, with which to ascertain the effects of curriculum design changes. In this paper, we present our approach of analyzing software repositories in order to gauge student behavior during project work. We evaluate this approach in a case study of a university undergraduate software development course teaching agile development methodologies. Surveys revealed positive attitudes towards the course and the change of employed development methodology from Scrum to Kanban. However, surveys were not usable to ascertain the degree to which students had adapted their workflows and whether they had done so in accordance with course goals. Therefore, we analyzed students' software repository data, which represents information that can be collected by educators to reveal insights into learning successes and detailed student behavior. We analyze the software repositories created during the last five courses, and evaluate differences in workflows between Kanban and Scrum usage.}, author = {Matthies, Christoph and Teusner, Ralf and Hesse, Guenter}, booktitle = {IEEE Frontiers in Education Conference (FIE)}, keywords = {development_artifacts software_engineering Kanban Scrum Education}, month = {oct}, pages = {1-9}, title = {Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts}, year = 2018 }
Hesse, G., Reissaus, B., Matthies, C., Lorenz, M., Kraus, M., Uflacker, M.: Senska - Towards an Enterprise Streaming Benchmark.TPC Technology Conference (TPCTC). pp. 25-40. , Cham (2018).
[ BibTeX ] [ Download ]

@inproceedings{hesse2018senska, abstract = {In the light of growing data volumes and continuing digitization in fields such as Industry 4.0 or Internet of Things, data stream processing have gained popularity and importance. Especially enterprises can benefit from this development by augmenting their vital, core business data with up-to-date streaming information. Enriching this transactional data with detailed information from high-frequency data streams allows answering new analytical questions as well as improving current analyses, e.g., regarding predictive maintenance. Comparing such data stream processing architectures for use in an enterprise context, i.e., when combining streaming and business data, is currently a challenging task as there is no suitable benchmark.}, address = {Cham}, author = {Hesse, Guenter and Reissaus, Benjamin and Matthies, Christoph and Lorenz, Martin and Kraus, Milena and Uflacker, Matthias}, booktitle = {TPC Technology Conference (TPCTC)}, keywords = {esb benchmarking data_stream_processing senska}, pages = {25-40}, title = {Senska - Towards an Enterprise Streaming Benchmark}, type = {Publication}, year = 2018 }
Lorenz, M., Rudolph, J.-P., Hesse, G., Uflacker, M., Plattner, H.: Object-Relational Mapping Revisited - A Quantitative Study on the Impact of Database Technology on O/R Mapping Strategies.Hawaii International Conference on System Sciences (HICSS). pp. 4877-4886 (2017).
[ BibTeX ]

@inproceedings{lorenz2017objectrelational, author = {Lorenz, Martin and Rudolph, Jan-Peer and Hesse, Guenter and Uflacker, Matthias and Plattner, Hasso}, booktitle = {Hawaii International Conference on System Sciences (HICSS)}, keywords = {in-memory_database}, pages = {4877-4886}, title = {Object-Relational Mapping Revisited - A Quantitative Study on the Impact of Database Technology on O/R Mapping Strategies}, year = 2017 }
Hesse, G., Matthies, C., Reissaus, B., Uflacker, M.: A New Application Benchmark for Data Stream Processing Architectures in an Enterprise Context: Doctoral Symposium.ACM International Conference on Distributed and Event-based Systems (DEBS). pp. 359-362 (2017).
[ BibTeX ] [ Download ]

@inproceedings{Guenter2017a, abstract = {Against the backdrop of ever-growing data volumes and trends like the Internet of Things (IoT) or Industry 4.0, Data Stream Processing Systems (DSPSs) or data stream processing architectures in general receive a greater interest. Continuously analyzing streams of data allows immediate responses to environmental changes. A challenging task in that context is assessing and comparing data stream processing architectures in order to identify the most suitable one for certain settings. The present paper provides an overview about performance benchmarks that can be used for analyzing data stream processing applications. By describing shortcomings of these benchmarks, the need for a new application benchmark in this area, especially for a benchmark covering enterprise architectures, is highlighted. A key role in such an enterprise context is the combination of streaming data and business data, which is barely covered in current data stream processing benchmarks. Furthermore, first ideas towards the development of a solution, i.e., a new application benchmark that is able to fill the existing gap, are depicted.}, author = {Hesse, Guenter and Matthies, Christoph and Reissaus, Benjamin and Uflacker, Matthias}, booktitle = {ACM International Conference on Distributed and Event-based Systems (DEBS)}, keywords = {Internet_of_Things esb Performance_Benchmarking Stream_Processing Benchmark_Development Data_Stream_Processing}, pages = {359-362}, series = {DEBS '17}, title = {A New Application Benchmark for Data Stream Processing Architectures in an Enterprise Context: Doctoral Symposium}, year = 2017 }
Lorenz, M., Hesse, G., Rudolph, J.-P.: Object-relational Mapping Revised - Guideline Review and Consolidation.International Joint Conference on Software Technologies (ICSOFT). pp. 157-168 (2016).
[ BibTeX ]

@inproceedings{Martin2016a, author = {Lorenz, Martin and Hesse, Guenter and Rudolph, Jan-Peer}, booktitle = {International Joint Conference on Software Technologies (ICSOFT)}, keywords = {imported}, pages = {157-168}, title = {Object-relational Mapping Revised - Guideline Review and Consolidation}, year = 2016 }
Hesse, G., Lorenz, M.: Conceptual Survey on Data Stream Processing Systems.IEEE International Conference on Parallel and Distributed Systems (ICPADS).797-802 (2015).
[ BibTeX ] [ Download ]

@article{Guenter2015a, author = {Hesse, Guenter and Lorenz, Martin}, journal = {IEEE International Conference on Parallel and Distributed Systems (ICPADS)}, keywords = {Storm esb Flink Stream_Processing Samza Spark}, pages = {797-802}, title = {Conceptual Survey on Data Stream Processing Systems}, year = 2015 }