Scaling Beyond DRAM with PMem without Compromising Performance

在不影响性能的情况下使用 PMem 扩展到 DRAM 之外

• 批准号: 361477420
• 负责人: Professor Dr. Alfons Kemper, Ph.D.
• 金额: --
• 依托单位: Lehrstuhl für Informatik III: Datenbanksysteme
• 依托单位国家: 德国
• 项目类别: Priority Programmes
• 财政年份: 2017
• 资助国家: 德国
• 起止时间: 2016-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/361477420?language=en
• 关键词: Scaling; Beyond; DRAM; PMem; without

Over the last decade our group has focused on developing in-memory database technology, most notably the HyPer database. The increases in main memory capacities over the last two decades have made in-memory data processing feasible while offering unprecedented performance. Even though most transactional data fits into the largest scale-up servers that facilitate several TB of DRAM capacity, we should reconsider the new opportunities of persistent memory (PMem) for two reasons: (1) its cost/performance ratio appears beneficial in comparison to pure DRAM systems and (2) new applications in the Big Data era require even higher data volumes. Other developments try to tackle the latter with scale-out solutions. However, they incur a much higher communication overhead – even when using new costly communication technologies like InfiniBand and RDMA. As an alternative, we want to scale beyond DRAM capacity by exploiting emerging PMem capabilities in scale-up servers without compromising the pure in-memory processing performance (for those working sets that fit into the large capacity DRAM and PMem). Working sets beyond DRAM plus PMem capacity should induce only a graceful degradation by relying on fast SSD storage. The research work proposed here shall be integrated into the Umbra database system. Umbra is the "spiritual" successor and evolution of our successful pure in-memory system HyPer. It completely eliminates the restrictions of HyPer that data fits into main memory. Umbra is a long-term research project with many Big Data applications. In this three-year project, we concentrate on how modern storage systems such as persistent memory can be integrated as a first-class citizen of Umbra. This requires to re-design the storage- and index-structures for data as well as key functional components like buffering, logging and recovery. The overall goal that we want to achieve is scaling beyond costly DRAM capacity without slowing down if the working set fits into DRAM plus PMem and gracefully degrading if it grows beyond. The full-fledged database system prototype Umbra serves as an integration platform for researching the spectrum of possibilities for the incorporation of persistent memory into a database system. Thereby, Umbra constitutes a realistic (and invaluable) test bed for an end-to-end investigation to achieve and prove practical relevance of the foundational research. The (almost unique) broad expertise of the PIs’ groups covering database as well as OS experience (after Prof. Jana Giceva joined the TUM group) will facilitate the exploitation of the modern storage devices most effectively – either at DBMS or "deeper" and more generic at the OS level.

在过去的十年里，我们的团队一直专注于开发内存数据库技术，最著名的是HyperPer数据库。在过去的二十年中，主存储器容量的增加使得内存数据处理变得可行，同时提供前所未有的性能。尽管大多数事务数据都适合最大的可扩展服务器，可提供数TB的DRAM容量，但我们应该重新考虑持久性内存（PMem）的新机会，原因有两个：（1）与纯DRAM系统相比，其成本/性能比似乎是有益的;（2）大数据时代的新应用程序需要更高的数据量。其他发展试图通过横向扩展解决方案来解决后者。然而，它们会产生更高的通信开销-即使使用新的昂贵通信技术，如InfiniBand和RDMA。作为替代方案，我们希望通过在扩展服务器中利用新兴的PMem功能来扩展DRAM容量，而不会影响纯内存处理性能（对于适合大容量DRAM和PMem的工作集）。 超过DRAM加上PMem容量的工作集应该只会通过依赖快速SSD存储而引起适度的降级。本文提出的研究工作应纳入Umbra数据库系统。Umbra是我们成功的纯内存系统Hyper的“精神”继承者和进化。它完全消除了Hyper的限制，数据适合主内存。Umbra是一个长期的研究项目，有许多大数据应用程序。在这个为期三年的项目中，我们专注于如何将持久性内存等现代存储系统集成为Umbra的一等公民。这需要重新设计数据的存储和索引结构，以及缓冲、日志记录和恢复等关键功能组件。我们希望实现的总体目标是扩展到成本高昂的DRAM容量之外，如果工作集适合DRAM和PMem，则不会放慢速度，如果超出了，则会适度降级。成熟的数据库系统原型Umbra作为一个集成平台，用于研究将持久内存纳入数据库系统的可能性。因此，Umbra构成了一个现实的（和宝贵的）测试平台，用于端到端的调查，以实现和证明基础研究的实际相关性。PI小组（几乎是独一无二的）广泛的专业知识，涵盖数据库以及操作系统经验（在Jana Giceva教授加入TUM小组之后），将促进最有效地利用现代存储设备-无论是在DBMS还是在操作系统层面上“更深入”和更通用。