Our paper "Towards A GPU-Accelerated Stream Processing Engine Through Query Compilation" by Florian Schmeller, Dwi P. A. Nugroho (TU Berlin), Steffen Zeuch (TU Berlin) and Tilmann Rabl was accepted at LWDA '24 (co-located with KI 2024).
Abstract:
Over the last decade, data stream processing has emerged to provide real-time insights into large, unbounded volumes of data. At the same time, graphics processing units (GPU) have become an important accelerator for improving the performance of compute-bound applications. Nevertheless, state-of-the-art data streaming systems opt to scale-out and typically do not make efficient use of the underlying hardware. Recent work has shown that query compilation is a viable technique to support hardware advancements in query processing engines. However, it often comes with high development and maintenance costs. In particular, when the process involves hardware accelerators such as GPUs.
In this paper, we propose a framework for compiling data stream queries to efficient GPU code in a developer-friendly manner. We demonstrate the feasibility of our framework by integrating it into the data management system NebulaStream. Our experiments show that frequent memory transfers between CPU and GPU impact the query processing throughput.
