Next-Generation Data Management Systems and Software Tools

点击翻译按钮获取中文摘要

Modern data-intensive applications, such as machine learning and data mining, fundamentally depend on database systems to meet their performance, functionality and reliability requirements. This puts pressing needs on database systems to best utilize the hardware infrastructure. The recent commoditization of persistent memory (PM) and programmable networks is transforming both on-premise and cloud infrastructure. They bring many opportunities to optimize database systems, but also invalidate many prior designs and assumptions. Existing designs are not ready and will lead to sub-optimal results. This research program explores key principles for building next-generation database systems in the context of PM and programmable networks through the following three modules. 1. Persistent Memory Database System: PM fills the gap between main memory and storage, thus has the potential of enabling an "ideal" database system that commits transactions fast, recovers instantly, and performs as fast as today's main-memory databases but at a lower cost. Existing work often has to trade off some of these features for performance. This research leverages the diversity of PM (e.g., NVDIMMs, Intel Optane) to realize the aforementioned desirable functionality without tradeoffs. 2. Database-Network-PM Co-design: Future servers will be equipped with large amounts of PM and interconnected by programmable networks. This combination opens up opportunities for applications to offload operations to the network with immediate persistence by PM, freeing up precious CPU cores for more useful work. It requires a coordinated effort to co-design components in the network, use of PM and database system components to reap the full benefits of such hardware, which is the objective of this research. 3. Efficient and Reliable Tools for Programming PM and Fast Networks: The making of database systems relies on tools suitable for persistent memory and future programmable networks. But existing tools fall short on data integrity, reliability and programmability. These problems must be solved before PM and programmable networks can be widely adopted. This research focuses on such issues, with a final goal of creating and maintaining a framework and tools that evolve with future PM and programmable network technologies. Impact: With the recent commoditization, PM and programmable networks will likely become part of the standard infrastructure for data-intensive applications. This research program is necessary for database systems and tools to continue to evolve with hardware advances. It will enhance Canada's strengths in related areas and benefit applications that rely on database systems, e.g., data mining, machine learning and visualization, with high-performance yet low-cost solutions. It will contribute to Canadian economy by training highly qualified personnel (including female students and under-represented groups) whose expertise are in high demand in academia and industry.