Next-Generation Data Management Systems and Software Tools

现代数据密集型应用，如机器学习和数据挖掘，从根本上依赖于数据库系统来满足其性能、功能和可靠性要求。这就迫切需要数据库系统最好地利用硬件基础设施。最近持久内存（PM）和可编程网络的商品化正在改变内部部署和云基础设施。它们为优化数据库系统带来了许多机会，但也使许多先前的设计和假设无效。现有的设计还没有准备好，将导致次优结果。本研究计划通过以下三个模块探索在PM和可编程网络环境中构建下一代数据库系统的关键原则。1. 持久内存数据库系统：PM填补了主内存和存储之间的空白，因此有可能实现一个“理想的”数据库系统，该系统可以快速提交事务，立即恢复，并且执行速度与今天的主内存数据库一样快，但成本更低。现有的工作通常不得不为了性能而牺牲其中的一些特性。这项研究利用了PM的多样性（例如，nvdimm, Intel Optane）来实现上述理想的功能而无需权衡。2. 数据库-网络-PM协同设计：未来的服务器将配备大量的PM，并通过可编程网络相互连接。这种组合为应用程序提供了机会，可以通过PM将操作卸载到具有即时持久性的网络上，从而释放宝贵的CPU内核，用于更有用的工作。它需要一个协调的努力来共同设计网络中的组件，使用PM和数据库系统组件来获得这些硬件的全部好处，这是本研究的目标。3. 高效可靠的PM和快速网络编程工具：数据库系统的构建依赖于适合持久内存和未来可编程网络的工具。但现有工具在数据完整性、可靠性和可编程性方面存在不足。在PM和可编程网络得到广泛应用之前，必须解决这些问题。本研究关注这些问题，最终目标是创建和维护一个框架和工具，这些框架和工具将随着未来的PM和可编程网络技术的发展而发展。影响：随着最近的商品化，PM和可编程网络将可能成为数据密集型应用程序的标准基础设施的一部分。这个研究项目对于数据库系统和工具随着硬件的进步而不断发展是必要的。它将增强加拿大在相关领域的优势，并使依赖数据库系统的应用受益，例如数据挖掘、机器学习和可视化，并提供高性能且低成本的解决方案。它将通过培训学术界和工业界急需的高素质人才（包括女学生和代表性不足的群体），为加拿大经济作出贡献。

Modern data-intensive applications, such as machine learning and data mining, fundamentally depend on database systems to meet their performance, functionality and reliability requirements. This puts pressing needs on database systems to best utilize the hardware infrastructure. The recent commoditization of persistent memory (PM) and programmable networks is transforming both on-premise and cloud infrastructure. They bring many opportunities to optimize database systems, but also invalidate many prior designs and assumptions. Existing designs are not ready and will lead to sub-optimal results. This research program explores key principles for building next-generation database systems in the context of PM and programmable networks through the following three modules. 1. Persistent Memory Database System: PM fills the gap between main memory and storage, thus has the potential of enabling an "ideal" database system that commits transactions fast, recovers instantly, and performs as fast as today's main-memory databases but at a lower cost. Existing work often has to trade off some of these features for performance. This research leverages the diversity of PM (e.g., NVDIMMs, Intel Optane) to realize the aforementioned desirable functionality without tradeoffs. 2. Database-Network-PM Co-design: Future servers will be equipped with large amounts of PM and interconnected by programmable networks. This combination opens up opportunities for applications to offload operations to the network with immediate persistence by PM, freeing up precious CPU cores for more useful work. It requires a coordinated effort to co-design components in the network, use of PM and database system components to reap the full benefits of such hardware, which is the objective of this research. 3. Efficient and Reliable Tools for Programming PM and Fast Networks: The making of database systems relies on tools suitable for persistent memory and future programmable networks. But existing tools fall short on data integrity, reliability and programmability. These problems must be solved before PM and programmable networks can be widely adopted. This research focuses on such issues, with a final goal of creating and maintaining a framework and tools that evolve with future PM and programmable network technologies. Impact: With the recent commoditization, PM and programmable networks will likely become part of the standard infrastructure for data-intensive applications. This research program is necessary for database systems and tools to continue to evolve with hardware advances. It will enhance Canada's strengths in related areas and benefit applications that rely on database systems, e.g., data mining, machine learning and visualization, with high-performance yet low-cost solutions. It will contribute to Canadian economy by training highly qualified personnel (including female students and under-represented groups) whose expertise are in high demand in academia and industry.