Prof. Dr. Tilmann Rabl

Ryan Marcus

Affiliation: University of Pennsylvania
Title: Machine Learning Powered Query Optimizers



Database management systems (DBMSes) depend on query optimizers to transform a user's declarative query into an efficient execution plan. Query optimizers are critical because a bad query plan can be orders of magnitude slower than the optimal plan. Modern query optimizers are complex and expensive to maintain, as they integrate a wide range of hand-tuned heuristics and manually-engineered cost models which must be updated for every new capability added to the DBMS. I will present two recent approaches to query optimization that leverage deep reinforcement learning to simultaneously improve query performance and decrease maintenance burden. The first approach, Bao (SIGMOD '21), learns to "steer" an existing query optimizer by training an agent via a contextual multi-armed bandit framework. The second approach, Kepler (SIGMOD '23), aims to find optimal query plans for parameterized queries using an exhaustive search process. More broadly, both Bao and Kepler highlight the huge potential impact of applying machine learning to systems problems, giving us a glimpse of what a fully learned system could do, as well as highlighting several potential pitfalls along the way.

Short CV

Ryan Marcus is an assistant professor at the University of Pennsylvania, where he researches learned systems. Ryan focuses on the potential of machine learning to underpin the next generation of data management systems, especially query optimization, data storage, and indexing. Ryan received his PhD from Brandeis University, where he studied machine learning techniques for automating cloud data management systems. Ryan is also a scientist at Intel Labs and an avid World of Warcraft player.