EAGER: Scaling the Preprocessor and Making it More Intelligent in Deterministic Database Systems

本研究旨在弥合当前现实与数据库系统部署在数据中心的大型服务器集群或云中的大量虚拟机上的潜力之间的差距。目前还不存在可伸缩的、弹性的、兼容acid的数据库系统实现。通常，需要弹性可伸缩性的应用程序被迫围绕数据库系统缺乏ACID保证进行编程，而且许多应用程序太复杂，无法重写以解决这些问题。本项目的目标是通过以下方法克服这些问题：(1)采用创新的确定性架构实现数据库系统，保证非确定性处理事件不会影响数据库状态；(2)利用这种新架构避免集群中分布式事务的“提交协议”；(3)为确定性数据库设计一个可扩展的预处理器，用于收集、分析和分配事务到数据库集群，以进一步提高可伸缩性。(4)开发一种新的延迟事务评估方法，以分散负载并避免数据库负载峰值的破坏性影响。总的来说，这项研究使为许多不同用例（如电子商务、电信和在线拍卖）编写的数千个应用程序能够“免费”地实现可伸缩性，而无需重写应用程序代码。这项研究涉及博士生和本科生，并大力鼓励本科生参与研究。这项研究的开源代码、出版物和技术报告将通过项目网站http://db.cs.yale.edu/determinism/发布。

This research aims to bridge the gap between the current reality and the potential for database system deployments on large clusters of servers in a data center or large numbers of virtual machines in the cloud. There does not exist a scalable, elastic, ACID-compliant database system implementation today. In general, applications that require elastic scalability are forced to program around the lack of ACID guarantees of the database system, and many applications are too complicated to be rewritten to work around these issues. The goal of this project is to overcome these issues using the following approaches: (1) Implementing a database system using an innovative deterministic architecture that guarantees that nondeterministic processing events will not affect database state, (2) Leveraging this new architecture to avoid "commit protocols" for distributed transactions in a cluster, (3) Designing a scalable preprocessor for the deterministic database that collects, analyzes, and dispatches transactions to the database cluster in order to further improve scalability, and (4) Developing a new lazy transaction evaluation approach in order to spread out load and avoid damaging effects of database load spikes. Overall, this research enables thousands of applications written for many different use-cases (such as e-commerce, telecommunications, and online auctions) to achieve scalability "for free" without having to rewrite the application code. This research involves both Ph.D. students and undergraduates, with significant outreach efforts to encourage undergraduates to get involved in research. Open source code, publications, and technical reports from this research will be disseminated via the project web site http://db.cs.yale.edu/determinism/.