CAREER: Self-Driving Database Management Systems

在过去的四十年中，研究人员和供应商都构建了咨询工具来帮助人类管理员进行数据库管理系统（DBMS）调优和物理设计。然而，所有这些之前的工作都是不完整的，因为它们需要人类对任何变化做出最终决定，而且它们是在问题发生后解决问题的反动措施。真正的“自驾车”DBMS所需要的是一种明确为自主操作而设计的新软件体系结构。这样，DBMS就不需要人类来监督和维护软件了。它还支持对现代高性能dbms非常重要的新优化，但由于管理这些系统的复杂性已经超过了人类专家的能力，因此目前还不可能实现这些优化。这样一个系统将消除部署数据库的人力资本障碍，并允许社会各个方面的组织（例如，商业、科学、政府）更容易地从数据驱动的决策应用程序中获益。作为本研究的一部分开发的技术也适用于其他问题领域，其中自主操作可以提高软件系统的性能和效率，包括大型系统（例如，分布式dbms，数据中心）和小型设备（例如，移动设备，物联网传感器）。该项目研究了自动驾驶dbms的技术，这些技术结合了来自数据库系统、机器学习（ML）和控制理论的最新方法。由于ML中算法的进步，以及存储和计算硬件的改进，在DBMS中实现自治操作现在是可能的。与以前的尝试不同的是，系统的所有方面都由一个综合规划组件控制，该组件不仅针对当前的工作负载优化系统，而且在未来的工作负载趋势发生之前预测它们，以便系统能够相应地做好准备。这项工作将产生在线方法，用于发现基于这些工作负荷预测模型的相关优化操作，从而使计划组件能够以更少的训练数据收敛到更好的配置。除此之外，本研究还将研究如何在不影响DBMS性能（例如，由于重启而停机）或导致错误行为的情况下部署这些操作。结果将是一组有效的自驾车DBMS体系结构的第一原则，可以部署这些修改并为其集成模型提供必要的反馈。这些原则在识别抑制现有系统自动化和影响未来DBMS体系结构设计的问题时是及时的。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Over the last four decades, both researchers and vendors have built advisory tools to assist human administrators in database management system (DBMS) tuning and physical design. All of this previous work, however, is incomplete because they require humans to make the final decisions about any changes and they are reactionary measures that fix problems after they occur. What is needed for a truly "self-driving" DBMS is a new software architecture that is explicitly designed for autonomous operation. With this, the DBMS will remove the need for humans to oversee and maintain the software. It also enables new optimizations that are important for modern high-performance DBMSs, but which are not possible today because the complexity of managing these systems has surpassed the abilities of human experts. Such a system will remove the human capital impediments of deploying databases and allow organizations in all facets of society (e.g., business, science, government) to more easily derive the benefits of data-driven decision-making applications. The techniques developed as part of this research are also applicable to other problem domains where autonomous operation could improve a software system's performance and efficiency, including both larger systems (e.g., distributed DBMSs, data centers) and smaller devices (e.g., mobile devices, IoT sensors).This project investigates techniques for self-driving DBMSs that combines state-of-the-art methods from database systems, machine learning (ML), and control theory. Achieving autonomous operation in a DBMS is now possible due to algorithmic advancements in ML, as well as improvements to storage and computation hardware. What makes this different than earlier attempts is that all aspects of the system are controlled by an integrated planning component that not only optimizes the system for the current workload but also predicts future workload trends before they occur so that the system can prepare itself accordingly. This work will produce on-line methods for discovering relevant optimization actions based on these workload forecast models, thereby enabling the planning component to converge to a better configuration with less training data. In addition to this, this research will study how to deploy these actions without hindering the DBMS's performance (e.g., downtime due to restarts) or causing incorrect behavior. The outcome will be a set of first principles for efficient self-driving DBMS architectures that can deploy these modifications and provide the necessary feedback for their integrated models. Such principles are timely in identifying issues that inhibit automation of existing systems and influencing the design of future DBMS architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.