CAREER: Self-Driving Database Management Systems

在过去的四十年里，研究人员和供应商都建立了咨询工具，以帮助人类管理员进行数据库管理系统（DBMS）调优和物理设计。然而，所有这些先前的工作都是不完整的，因为它们需要人类对任何更改做出最终决定，并且它们是在问题发生后修复问题的反动措施。真正的“自动驾驶”DBMS所需要的是一种新的软件架构，它明确地为自主操作而设计。有了这个，DBMS将消除人类监督和维护软件的需要。它还实现了对现代高性能DBMS非常重要的新优化，但这些优化在今天是不可能实现的，因为管理这些系统的复杂性已经超过了人类专家的能力。这样的系统将消除部署数据库的人力资本障碍，并允许社会各个方面的组织（例如，商业、科学、政府），更容易从数据驱动的决策应用程序中获益。作为本研究的一部分开发的技术也适用于其他问题领域，其中自主操作可以提高软件系统的性能和效率，包括较大的系统（例如，分布式DBMS、数据中心）和较小的设备（例如，移动的设备，物联网传感器）。该项目研究了自驱动DBMS的技术，该技术结合了数据库系统，机器学习（ML）和控制理论的最先进方法。由于ML的算法进步以及存储和计算硬件的改进，现在可以在DBMS中实现自主操作。这与以前的尝试不同的是，系统的所有方面都由一个综合规划组件控制，该组件不仅针对当前的工作量优化系统，而且还在未来的工作量趋势发生之前预测它们，以便系统可以相应地做好准备。这项工作将产生在线方法，用于根据这些工作量预测模型发现相关的优化行动，从而使规划部分能够以较少的训练数据收敛到更好的配置。除此之外，本研究还将研究如何在不影响DBMS性能的情况下部署这些操作（例如，由于重新启动导致的停机时间）或导致不正确的行为。其结果将是一组有效的自驱动DBMS架构的首要原则，这些架构可以部署这些修改并为其集成模型提供必要的反馈。这些原则是及时识别问题，抑制现有系统的自动化和影响未来的DBMS架构的设计。这个奖项反映了NSF的法定使命，并已被认为是值得的支持，通过评估使用基金会的知识价值和更广泛的影响审查标准。

Over the last four decades, both researchers and vendors have built advisory tools to assist human administrators in database management system (DBMS) tuning and physical design. All of this previous work, however, is incomplete because they require humans to make the final decisions about any changes and they are reactionary measures that fix problems after they occur. What is needed for a truly "self-driving" DBMS is a new software architecture that is explicitly designed for autonomous operation. With this, the DBMS will remove the need for humans to oversee and maintain the software. It also enables new optimizations that are important for modern high-performance DBMSs, but which are not possible today because the complexity of managing these systems has surpassed the abilities of human experts. Such a system will remove the human capital impediments of deploying databases and allow organizations in all facets of society (e.g., business, science, government) to more easily derive the benefits of data-driven decision-making applications. The techniques developed as part of this research are also applicable to other problem domains where autonomous operation could improve a software system's performance and efficiency, including both larger systems (e.g., distributed DBMSs, data centers) and smaller devices (e.g., mobile devices, IoT sensors).This project investigates techniques for self-driving DBMSs that combines state-of-the-art methods from database systems, machine learning (ML), and control theory. Achieving autonomous operation in a DBMS is now possible due to algorithmic advancements in ML, as well as improvements to storage and computation hardware. What makes this different than earlier attempts is that all aspects of the system are controlled by an integrated planning component that not only optimizes the system for the current workload but also predicts future workload trends before they occur so that the system can prepare itself accordingly. This work will produce on-line methods for discovering relevant optimization actions based on these workload forecast models, thereby enabling the planning component to converge to a better configuration with less training data. In addition to this, this research will study how to deploy these actions without hindering the DBMS's performance (e.g., downtime due to restarts) or causing incorrect behavior. The outcome will be a set of first principles for efficient self-driving DBMS architectures that can deploy these modifications and provide the necessary feedback for their integrated models. Such principles are timely in identifying issues that inhibit automation of existing systems and influencing the design of future DBMS architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.