System Infrastructure for Data-Centric ML Pipelines (Matthias Böhm)

Summary written by Klara Prigge

The lecture “System Infrastructure for Data-Centric ML Pipelines” was delivered by Matthias Böhm, a professor at TU Berlin and the creator as well as the lead maintainer of Apache SystemDS, an open-source platform for large-scale machine learning. The session focused on data-centric machine learning (ML) pipelines, exploring both the system infrastructure needed for ML and the research his group is pursuing in automating and optimizing these pipelines.

The Importance of Data in ML Pipelines

Classical ML pipelines focus heavily on algorithmic improvements like tweaking models and hyperparameter and most of the times have some additional data pre-processing steps. However, Böhm emphasizes that the concept of data-centric ML pipelines follows the premise that a high-quality and good coverage dataset is actually much more important than algorithmic details of the actual models that we are training.
Data-centric ML pipelines essentially wrap the pipeline in an additional outer loop for all kinds of data related tasks like for example data integration & cleaning, data programming & augmentation, validation & debugging, deployment & scoring and also data engineering and data preparation.

The key insight is that state-of-the-art ML primitives themselves are often based on machine learning itself, making it natural to build pipeline infrastructure on top of ML systems.

The ML System Perspective

While ML applications like classification, regression, and clustering are widely known, Böhm underscores a broader view:

An ML system encompasses everything from applications to hardware:

• Algorithms and models

• Compilers and runtime optimizations

• Distributed execution frameworks

• Hardware accelerators

This end-to-end view enables researchers to optimize across layers, from high-level ML abstractions down to low-level tensor computations.

Figure 1 An ML System - in the broad and narrow scope (figure taken from Matthias Böhm's slides)

Optimizing Through Tensor Computations

Many ML operations regarding data science, query processing and simulation primitives can be expressed as tensor operations (e.g., matrix multiplication and aggregation). This provides several advantages in terms of simplicity as hardware vendors already provide highly optimized libraries for linear algebra operations so with a mapping one can reuse those highly tuned kernels.

An example for a data cleaning primitive that can be mapped to such tensor computations is for example the detection of robust functional dependencies, so functional dependencies where the dependency must be fulfilled for a certain threshold of data points to be counted as valid. This can be used to fill missing values, but also to clean up data errors. More complex examples in the research of Böhm’s chair includes multivariate imputation by chained equations (MICE) that is also used for data cleaning by using the existing data as training data to train classifiers and regressors to then impute individual missing values. Then they take the target ML application and all other constraints and perform a search of first logical pipelines of cleaning primitives to find out in which order, and with which hyper parameters it should be assembled. After that the end-to-end pipelines are tuned and finally run in parallel computation.

Figure 2 MICE workflow (figure taken from Matthias Böhm's slides)

He also presents other examples, including how Model debugging can be optimized with SliceLine. SliceLine identifies subpopulations where a model performs poorly by efficiently exploring and pruning a large space of candidate feature slices. It uses sparse linear-algebra operations to scale this search, helping practitioners pinpoint data or fairness issues and guide targeted improvements. He further notes that synthetic data augmentation can similarly leverage linear-algebra–based transformations to expand and enrich training data

Data Independence & Heterogeneity

Böhm then shifted the lecture toward one of the most persistent challenges in modern ML systems: dealing with the immense heterogeneity that characterizes both data and hardware today. Böhm stresses that modern ML systems must handle heterogeneous data representations like dense, sparse and compressed data representations, multiple hardware accelerators with varying performance and energy trade-offs and diverse data types and precision formats. Inspired by Hellerstein’s Principle, his team argues for declarative infrastructure that separates model logic from hardware and data layout. They implement this within Apache SystemDS, offering a domain-specific language (DSL) with R-like syntax and a compiler that produces optimized runtime plans for both local in-memory and distributed (Spark) execution.

What Offers SystemDS?

SystemDS evolved from IBM’s SystemML and now supports:

• Linear algebra programs (matrix operations, element-wise ops, control flow)

• Built-in ML functions (e.g., linear regression, feature selection)

• Optimizing compiler for hybrid execution plans

Key research features at his group include multi-level lineage tracing for reproducibility and caching, compressed linear algebra with dictionary encoding, run-length encoding and workload-aware compression as well as federated learning, so training across distributed sites without centralizing data.

Advanced Optimization Techniques

Böhm then turned to several other advanced optimization techniques that push SystemDS beyond conventional ML execution. A focus was the handling of sparse data, which appears frequently in applications like recommendation systems or text processing. Sparse matrices pose challenges because naive execution often creates dense intermediate results, leading to large slowdowns. They avoid this by applying compiler passes that fuse operations directly on sparse structures, eliminating unnecessary materialization and achieving significant performance gains.

He also discussed ongoing work on energy- and carbon-aware execution. As ML pipelines grow, their energy footprint becomes a meaningful constraint. They experiment with integrating energy measurements into optimization process, enabling the compiler to prefer configurations that save energy while keeping accuracy and runtime within acceptable bounds.

Another theme was data-centric optimization to identify parts of the dataset where models fail, or where training data is unbalanced.

For example, within the developed framework Plutus, data lake sampling techniques are utilized and linked up with SliceLine where they try to identify what the slices are where the system makes a lot of mistakes. With buget-aware sampling strategies from data lakes they then pull exactly for those slices’ preferable instances and by that they tailor the distribution incrementally to train those models and improve the ML pipeline.

Figure 3 PLUTUS workflow (figure taken from Matthias Böhm's slides)

Other works in this direction include using LLMs to assist in pipeline generation and data cleaning by refining data sets in an iterative manner.

The lecture also addressed multimodal data. When ML pipelines involve text, images, audio, or time series simultaneously, finding the right representation and fusion strategy becomes difficult. They explore automated representation search that evaluates different encodings and combination strategies, reducing the burden on practitioners and adapting pipelines to diverse data types.

Another strand of Böhm’s research that again focuses on data-centric methods that improve datasets themselves rather than only improving models is the exploration of “core sets,” where the goal is to identify a small but high-quality subset of the training data that yields nearly the same accuracy as the full dataset.

Although many proposed strategies exist, Böhm noted that in practice they are surprisingly difficult to outperform compared to simple random sampling - though targeted techniques can still offer benefits.

His team is investigating how sampling and data augmentation can be combined into a unified strategy. Sampling aims to reduce the dataset while maintaining accuracy, whereas augmentation seeks to dramatically expand the amount of data to gain accuracy. Their approach looks for a middle ground: selecting representative samples while synthetically generating data only for underrepresented regions of the distribution.

Figure 4 Sampling vs. Data Augment (figure taken from Matthias Böhm's slides)

This work spans several domains, including histopathological cancer images and ECG time-series data, where domain-specific properties must be preserved. Related to this is their effort to build more informative evaluation datasets. Instead of relying on a single test set, they create test collections that vary along dimensions such as difficulty, distribution shifts, and subgroup coverage. This aligns with emerging regulatory expectations like the EU AI Act, which requires that training and evaluation data be representative for high-risk applications.

Böhm also discussed their work on adaptive model serving through learned sparsification and quantization. The idea is to maintain several model variants and to dynamically choose which to serve depending on latency and accuracy requirements. By continuously sampling real traffic and comparing the predictions of smaller models against a large reference model, the system can monitor accuracy degradation and decide when model swapping is safe. Although initially proposed as a vision paper, the group is now building a prototype that supports fast model swapping and shows promising early results.

Key Takeaways & Outlook

To summarize, Matthias Böhm argued that data-centric ML pipelines, focused on high-quality, well-covered datasets, often matter more than tweaking models. His team builds these pipelines on SystemDS, a DSL plus optimizing compiler that produces hybrid local/distributed execution plans and provides data independence across formats and hardware. A central insight is that many data-engineering, debugging and augmentation tasks can be expressed as tensor/linear-algebra operations, letting them reuse highly optimized matrix kernels. They automate cleaning and imputation (e.g., functional-dependency and MICE pipelines) and search/tune pipeline compositions much like hyperparameter tuning. SystemDS includes lineage tracing, workload-aware compression, and sparsity-exploiting fusion to enable caching, speedups, and memory-efficient execution. The project also supports federated training for privacy-sensitive scenarios and uses LLMs plus data catalogs to auto-generate effective preprocessing pipelines. Finally, they pursue representation search, learned augmentation/sampling, and model quantization/sparsification to trade off accuracy, latency, and energy in production.

Böhm concluded with an outlook that stresses compiler-driven optimization for heterogeneous hardware and sparse data, as well as deeper automation of data-centric pipelines for multimodal workloads. He highlighted the importance of energy- and resource-aware model serving as ML systems move into production. He also pointed to broad opportunities for interdisciplinary collaboration that combines ML systems, domain knowledge, and applied research:

“I firmly believe that in order to build the right system infrastructure, it's also about selective collaborations with actual kind of applications in order to not create artificial problems and solving them.”

Overall, his vision is a holistic ML system stack that tightly unifies algorithms, infrastructure, and real-world data.