Abstract: Deep learning (DL) is growing in popularity for many advanced data analytics applications in enterprise, Web, scientific, and other domains. Naturally, resource efficiency of DL systems and the productivity of their users are pressing challenges to democratizing DL. In this talk, I present a new technical direction from my research that tackles such challenges with a database-inspired lens: higher-level specification and multi-query optimization (MQO). By exploiting higher-level abstractions of DL usage already inherent in practice, I show how we can automatically restructure the underlying execution to improve resource efficiency, reduce runtimes and costs, and in turn, improve user productivity. To this end,
Our approach benefits both DL inference and training, as I illustrate with three recent systems: Vista , with MQO for CNN transfer learning; Krypton , with MQO for CNN inference; and Cerebro , with MQO for parallel DL model selection. All of our techniques are easily integrated with existing DL systems (eg, TensorFlow and PyTorch) without affecting their internal code, making practical adoption easier. I will conclude by highlighting some of our ongoing and upcoming work on generalizing Cerebro to more higher-level DL tasks and to more execution environments such as cloud-native settings.
Bio: Arun Kumar is an Assistant Professor in the Department of Computer Science and Engineering and the Halicioglu Data Science Institute at the University of California, San Diego. He is a member of the Database Lab and Center for Networked Systems and an affiliate member of the AI Group. His primary research interests are in data management and systems for machine learning / artificial intelligence-based data analytics. Systems and ideas based on his research have been released as part of the Apache MADlib open-source library, shipped as part of products from Cloudera, IBM, Oracle, and Pivotal, and used internally by Facebook, Google, LogicBlox, Microsoft, and other companies. He is a recipient of two SIGMOD research paper awards, a SIGMOD Research Highlight Award, three distinguished reviewer awards from SIGMOD / VLDB,
Research webpage: https://adalabucsd.github.io/