CAREER: Self-Driving Database Management Systems

职业：自动驾驶数据库管理系统

• 批准号: 1846158
• 负责人: Andrew Pavlo
• 金额: $ 49.41万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-02-15 至 2024-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1846158&HistoricalAwards=false
• 关键词: CAREER; Self; Driving; Database; Management

Over the last four decades, both researchers and vendors have built advisory tools to assist human administrators in database management system (DBMS) tuning and physical design. All of this previous work, however, is incomplete because they require humans to make the final decisions about any changes and they are reactionary measures that fix problems after they occur. What is needed for a truly "self-driving" DBMS is a new software architecture that is explicitly designed for autonomous operation. With this, the DBMS will remove the need for humans to oversee and maintain the software. It also enables new optimizations that are important for modern high-performance DBMSs, but which are not possible today because the complexity of managing these systems has surpassed the abilities of human experts. Such a system will remove the human capital impediments of deploying databases and allow organizations in all facets of society (e.g., business, science, government) to more easily derive the benefits of data-driven decision-making applications. The techniques developed as part of this research are also applicable to other problem domains where autonomous operation could improve a software system's performance and efficiency, including both larger systems (e.g., distributed DBMSs, data centers) and smaller devices (e.g., mobile devices, IoT sensors).This project investigates techniques for self-driving DBMSs that combines state-of-the-art methods from database systems, machine learning (ML), and control theory. Achieving autonomous operation in a DBMS is now possible due to algorithmic advancements in ML, as well as improvements to storage and computation hardware. What makes this different than earlier attempts is that all aspects of the system are controlled by an integrated planning component that not only optimizes the system for the current workload but also predicts future workload trends before they occur so that the system can prepare itself accordingly. This work will produce on-line methods for discovering relevant optimization actions based on these workload forecast models, thereby enabling the planning component to converge to a better configuration with less training data. In addition to this, this research will study how to deploy these actions without hindering the DBMS's performance (e.g., downtime due to restarts) or causing incorrect behavior. The outcome will be a set of first principles for efficient self-driving DBMS architectures that can deploy these modifications and provide the necessary feedback for their integrated models. Such principles are timely in identifying issues that inhibit automation of existing systems and influencing the design of future DBMS architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的四十年中，研究人员和供应商都建立了咨询工具，以帮助人类管理员进行数据库管理系统（DBMS）调整和物理设计。但是，所有这些先前的工作都是不完整的，因为他们要求人类就任何变化做出最终决定，并且是解决问题后解决问题的反动措施。真正的“自动驾驶” DBMS所需的是一种新的软件体系结构，是针对自动操作而设计的。这样，DBM将消除对人类监督和维护软件的需求。它还实现了对现代高性能DBMS很重要的新优化，但是今天是不可能的，因为管理这些系统的复杂性已经超过了人类专家的能力。这样的系统将消除部署数据库的人力资本障碍，并允许社会各个方面（例如商业，科学，政府）的组织更轻松地获得数据驱动的决策应用程序的好处。 The techniques developed as part of this research are also applicable to other problem domains where autonomous operation could improve a software system's performance and efficiency, including both larger systems (e.g., distributed DBMSs, data centers) and smaller devices (e.g., mobile devices, IoT sensors).This project investigates techniques for self-driving DBMSs that combines state-of-the-art methods from database systems, machine learning (ML), and control theory.由于ML中的算法进步以及对存储和计算硬件的改进，现在可以在DBMS中实现自主操作。与早期尝试不同的是，系统的所有方面都由集成的计划组件控制，该计划不仅可以优化当前工作负载的系统，而且还可以预测未来的工作负载趋势在发生之前，以便系统可以相应地准备自己。这项工作将产生在线方法，以根据这些工作量预测模型发现相关的优化操作，从而使计划组件能够通过更少的培训数据收敛到更好的配置。除此之外，这项研究还将研究如何在不阻碍DBMS的性能（例如由于重新启动引起的停机时间）或导致不正确行为的情况下部署这些动作。结果将是一组有效的自动驾驶DBMS架构的第一原则，这些原理可以部署这些修改并为其集成模型提供必要的反馈。此类原则及时确定了抑制现有系统自动化并影响未来DBMS架构设计的问题。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估评估来审查标准的评估。

项目成果

MB2: Decomposed Behavior Modeling for Self-Driving Database Management Systems

• DOI: 10.1145/3448016.3457276
• 发表时间: 2021-06
• 期刊: Proceedings of the 2021 International Conference on Management of Data
• 作者: Lin Ma;William Zhang;Jie Jiao;Wuwen Wang;Matthew Butrovich;Wan Shen Lim;Prashanth Menon;Andrew Pavlo

Make Your Database System Dream of Electric Sheep: Towards Self-Driving Operation

• DOI: 10.14778/3476311.3476411
• 发表时间: 2021-07
• 期刊: Proc. VLDB Endow.
• 作者: Andrew Pavlo;Matthew Butrovich;Lin Ma;Prashanth Menon;Wan Shen Lim;Dana Van Aken;William Zhang

External vs. Internal: An Essay on Machine Learning Agents for Autonomous Database Management Systems

• DOI: 10.37745/ejcsit.2013/vol10n52431
• 发表时间: 2022-05
• 期刊: IEEE Data Eng. Bull.
• 作者: Andrew Pavlo;Matthew Butrovich;Ananya Joshi;Lin Ma;Prashanth Menon;Dana Van Aken;Lisa Lee;R. Salakhutdinov

Mainlining databases: supporting fast transactional workloads on universal columnar data file formats

主线数据库：支持通用列式数据文件格式的快速事务工作负载

• DOI: 10.14778/3436905.3436913
• 发表时间: 2020
• 期刊: Proceedings of the VLDB Endowment
• 影响因子: 2.5
• 作者: Li, Tianyu;Butrovich, Matthew;Ngom, Amadou;Lim, Wan Shen;McKinney, Wes;Pavlo, Andrew
• 通讯作者: Pavlo, Andrew

Filter Representation in Vectorized Query Execution

• DOI: 10.1145/3465998.3466009
• 发表时间: 2021-06
• 期刊: Proceedings of the 17th International Workshop on Data Management on New Hardware
• 作者: Amadou Latyr Ngom;Prashanth Menon;Matthew Butrovich;Lin Ma;Wan Shen Lim;T. Mowry;Andrew Pavlo