Hybrid OLTP/OLAP Main Memory Database System Based on Virtual Memory Snapshots

在这个项目的第一阶段，我们开发了一个支持OLTP和OLAP应用程序的混合内存数据库系统。其关键思想是利用操作系统/处理器对虚拟内存管理的支持。这允许产生一致的数据库快照以将OLAP查询与OLTP事务隔离--即使它们共享同一数据库。基于项目开始时已经存在的第一个原型Hyper，并证明了这种OLTP和OLAP混合数据库的可行性，我们在项目过程中实现了一个完整的数据库系统。国际上最负盛名的场所(SIGMOD、ICDE、CIDR和VLDB)上的大量出版物表明，HYPER是最引人注目的学术数据库项目之一。这一说法也得到了ICDE 2014年度最佳论文奖以及享有盛誉的VLDB早期职业奖2014年T.Neumann在超级数据库引擎方面的工作的支持。在接下来的一段时间里，我们将重点关注以下主题：1)多版本并发控制方案是一种在不增加只读事务开销的情况下同步OLTP(Online Transaction Processing)事务的有效方法。此外，应该扩展以前在硬件事务内存上的工作，以同步新的自适应基树索引。2)到目前为止，我们一直专注于优化SQL 92查询的性能。对于决策支持查询，窗口函数构成了查询语言的重要扩展。到目前为止，还没有关于包含窗口函数的查询的大规模并行计算的研究，而窗口函数对于开发现代多核服务器的巨大聚合计算能力是必不可少的。3)在不久的将来，服务器将由不同的组件组成(GPU、节能CPU、高性能CPU等)。为了以节能和/或性能优化的方式利用这些组件，需要调度器将查询评估任务分配给最适合的硬件组件。4)硬件供应商(英特尔、惠普等)目前正在开发非易失性DRAM。这种新的存储技术特别适用于像Hyper这样的内存数据库系统，以减少恢复过程中的故障转移时间，从而提高可用性。这些主题中的每一个对于博士学位论文主题来说都足够复杂。现有的超级实施将被用作所有工程工作的实验平台，以证明其实际意义。

In the first phase of this project we developped a hybrid main memory database system that supports OLTP- and OLAP-applications. The key idea is to exploit the OS/processor-support for virtual memory management. This allows to spawn consistent database snapshots to isolate OLAP queries from the OLTP transactions -- even though they share the same database. Based on the first prototype, called HyPer, that existed already at the start of the project and proved the feasibility of such a hybrid OLTP&OLAP database, we realized in the course of the project a full-fledged database system. The numerous publications in the internationally most prestigious venues (SIGMOD, ICDE, CIDR, and VLDB) suggest that HyPer is one of the most visible academic database projects. This claim is also supported by the Best Paper Award of the ICDE 2014as well as the prestigious VLDB Early Career Award 2014 for T. Neumann's work on the HyPer database engine. In the continuation period we want to focus on the following topics:1) Multi-version concurrency control schemes are a promising method for synchronizing OLTP (online transaction processing) transactions without much overhead for read-only transactions. Furthermore, the previous work on hardware transactional memory should be extended for synchronizing the new adaptive radix-tree indexing.2) So far we have concentrated on optimizing the performance of SQL 92 queries. For decision support queries window functions constitute an important extension of the query language. So far no prior research can be found on massively parallel evalaution of queries containing window functions which is, however, essential to exploit the vast aggregated compute power of modern many-core servers.3) In the near future servers will consist of heterogeneous components (GPUs, energy-safing CPUs, high-performance CPUs, etc.) In order to utilize these components in an energy-efficient and/or performance-optimized way a scheduler is needed that allocates query evaluation tasks to the best-suited hardware component. 4) The hardware vendors (Intel, HP, etc.) are currently developing non-volatile DRAM. This new storage technology is particularly suitable for main-memory database systems like HyPer to reduce fail-over times during recovery and thus increase the availability. Each of these topics is sufficiently complex for a PhD dissertation topic. The existing HyPer implementation will be used as an experimental platform for all engineering work in order to prove its practical relevance.