III: Medium: Learning-based Synthesis of Data Processing Engines

现代数据处理系统被设计为通用系统，因为它们可以处理各种各样的应用程序和数据。不幸的是，这种通用的性质导致这些系统为每个应用程序和用户实现低于最佳的性能。相反，为了支持广泛的用例，必须做出技术上的妥协，这通常会导致性能比高度定制的系统所能实现的性能差几个数量级。同时，为每个单独的应用程序和用户从头开始开发数据库系统既不经济也不实用。该项目的目标是探索如何使用机器学习来自动定制特定应用程序或用户的数据库系统，以实现所谓的“实例优化”。 如果成功，该项目将改变支撑互联网和许多企业计算系统的现代数据库系统的构建方式，从而使系统具有更好的性能，或者能够使用比当前系统少得多的硬件来处理大型数据集。具体地说，该项目研究了学习模型在多大程度上可以自动实例优化大型数据处理系统的各个组件：1）数据索引，其中模型可以预测数据库中键的位置; 2）算法，包括排序和连接，其中模型可以预测记录应该在排序列表中的位置，或者连接元组在另一个关系中的位置; 3）优化器，其中模型可以预测用于处理对数据的查询的最佳计划，以及4）存储布局，其中模型可以预测针对特定查询工作负载的数据的最佳布局。这提出了一些智力上的深层次问题，包括什么类型的模型工作得最好，我们可以对这些模型的性能给出什么样的理论保证，这样生成的系统将如何与手动调整的系统进行比较，这样的系统如何利用新的硬件，如TPU/GPU，以及程序合成将如何与这样的建模数据一起工作，推进数据库，机器学习，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern data-processing systems are designed to be general-purpose systems, in that they can handle a wide variety of applications and data. Unfortunately, this general-purpose nature causes these systems to achieve below-optimal performance for every single application and user. Rather technical compromises have to be made to support a wide range of use cases, often leading to orders-of-magnitude worse performance than what a highly customized system would be able to achieve. At the same time, developing a database system from scratch for each individual application and user is neither economical nor practical. The goal of this project is to explore how machine learning can be used to automatically customize a database system for a specific application or user to achieve so called 'instance-optimality'. If successful, this project will transform the way that modern database systems that underpin the Internet and many enterprise computing systems are built, resulting in systems with much better performance or systems that are able to process large datasets using much less hardware than current systems. Concretely, the project investigates to what extent learned models can automatically instance-optimize the various components of a large-scale data processing system: 1) data indexing, where a model can predict the location of a key in a database; 2) algorithms, including sorting and joins, where a model can predict where in a sorted list a record should go, or where joining tuples are in another relation; 3) optimizers, where a model can predict the optimal plan to use for processing queries on data, and 4) storage layout, where a model can predict the optimal layout of data for a particular query workload. This raises a number of intellectually deep questions, including what types of models work best, what theoretical guarantees we can give about the performance of these models, how such generated systems will compare to hand-tuned systems, how such systems can exploit new hardware such as TPUs/GPUs and how program synthesis will work with such modelled data, advancing the fields of databases, machine learning, and program modeling and synthesis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.