Next-Generation Data Management Systems and Software Tools

现代数据密集型应用，如机器学习和数据挖掘，基本上依赖于数据库系统来满足其性能、功能和可靠性要求。这对数据库系统提出了最大限度地利用硬件基础设施的迫切需求。最近永久存储器(PM)和可编程网络的商品化正在改变本地和云基础设施。它们带来了许多优化数据库系统的机会，但也使许多先前的设计和假设失效。现有的设计还没有准备好，将导致次优结果。本研究计划通过以下三个模块探索在PM和可编程网络的背景下构建下一代数据库系统的关键原则。1.持久内存数据库系统：PM填补了主存和存储之间的空白，因此有可能实现一个提交事务快速、即时恢复、执行速度与当今的主存数据库一样快、但成本更低的“理想”数据库系统。现有的工作经常不得不牺牲其中的一些功能来换取性能。本研究利用PM的多样性(例如，NVDIMM、Intel Optane)来实现上述所需功能，而无需进行权衡。2.数据库-网络-PM协同设计：未来的服务器将配备大量PM，并通过可编程网络互联。这种组合为应用程序提供了机会，可以通过PM的即时持久性将操作分流到网络，从而释放宝贵的CPU核心用于更有用的工作。它需要协调努力来共同设计网络中的组件，使用PM和数据库系统组件来获得此类硬件的全部好处，这是本研究的目标。3.高效可靠的PM和快速网络编程工具：数据库系统的制作依赖于适用于永久存储器和未来可编程网络的工具。但现有工具在数据完整性、可靠性和可编程性方面存在不足。在PM和可编程网络能够被广泛采用之前，必须解决这些问题。这项研究的重点是这些问题，最终目标是创建和维护一个与未来的PM和可编程网络技术一起发展的框架和工具。影响：随着最近的商品化，PM和可编程网络很可能成为数据密集型应用程序的标准基础设施的一部分。这项研究计划对于数据库系统和工具继续随着硬件的进步而发展是必要的。它将增强加拿大在相关领域的优势，并以高性能但低成本的解决方案使依赖数据库系统的应用程序受益，例如数据挖掘、机器学习和可视化。它将通过培训高素质的人员(包括女学生和代表性不足的群体)，为加拿大经济做出贡献，这些人员的专业知识在学术界和工业界都很受欢迎。

Modern data-intensive applications, such as machine learning and data mining, fundamentally depend on database systems to meet their performance, functionality and reliability requirements. This puts pressing needs on database systems to best utilize the hardware infrastructure. The recent commoditization of persistent memory (PM) and programmable networks is transforming both on-premise and cloud infrastructure. They bring many opportunities to optimize database systems, but also invalidate many prior designs and assumptions. Existing designs are not ready and will lead to sub-optimal results. This research program explores key principles for building next-generation database systems in the context of PM and programmable networks through the following three modules. 1. Persistent Memory Database System: PM fills the gap between main memory and storage, thus has the potential of enabling an "ideal" database system that commits transactions fast, recovers instantly, and performs as fast as today's main-memory databases but at a lower cost. Existing work often has to trade off some of these features for performance. This research leverages the diversity of PM (e.g., NVDIMMs, Intel Optane) to realize the aforementioned desirable functionality without tradeoffs. 2. Database-Network-PM Co-design: Future servers will be equipped with large amounts of PM and interconnected by programmable networks. This combination opens up opportunities for applications to offload operations to the network with immediate persistence by PM, freeing up precious CPU cores for more useful work. It requires a coordinated effort to co-design components in the network, use of PM and database system components to reap the full benefits of such hardware, which is the objective of this research. 3. Efficient and Reliable Tools for Programming PM and Fast Networks: The making of database systems relies on tools suitable for persistent memory and future programmable networks. But existing tools fall short on data integrity, reliability and programmability. These problems must be solved before PM and programmable networks can be widely adopted. This research focuses on such issues, with a final goal of creating and maintaining a framework and tools that evolve with future PM and programmable network technologies. Impact: With the recent commoditization, PM and programmable networks will likely become part of the standard infrastructure for data-intensive applications. This research program is necessary for database systems and tools to continue to evolve with hardware advances. It will enhance Canada's strengths in related areas and benefit applications that rely on database systems, e.g., data mining, machine learning and visualization, with high-performance yet low-cost solutions. It will contribute to Canadian economy by training highly qualified personnel (including female students and under-represented groups) whose expertise are in high demand in academia and industry.