MxKernel: A Bare-Metal Runtime System for Database Operations on Heterogeneous Many-Core Hardware

新兴硬件平台的特点是高度并行、复杂的内存层次结构和日益增加的硬件异构性。他们的理论峰值数据处理性能只能被释放，如果不同的系统软件更紧密地合作，如果他们的传统的依赖关系和接口被重新设计。作为一种新的方法，我们已经开发的关键概念和一个原型实现的裸机运行时系统名为MxKernel（第一个资金周期的项目）。MxKernel为数据库管理系统（DBMS）和操作系统（OS）提供了非常轻量级的资源管理，它们都作为运行时之上的对等体运行。在MxKernel中，异构性和并行性成为一等公民，并且从一开始就考虑了深层内存层次结构。与传统的"线程"模型不同，MxKernel提供了一个更简单的控制流抽象：MxTasks模型封闭的工作单元，MxKernel将保证所需的执行语义，例如对内存中特定对象的独占访问。对于异构性和资源共享，它们也可以是一个非常优雅的抽象。此外，MxTasks还使用元数据进行注释，例如代码变体（以支持异构性）、内存访问行为（以提高缓存效率并支持内存层次结构）或MxTasks之间的依赖关系（以提高调度并避免同步成本）。由于所需的元数据是精确可用的，MxKernel可以提供一种轻量级的，但高效的资源管理形式，即使是跨应用程序，操作系统和数据库。基于MxKernel原型和我们有希望的结果，例如，通过明智的数据预取优化内存使用的能力，我们现正以更有系统的方式，探讨及发掘新系统软件架构的全部潜力。为此，我们将研究更现实的应用场景和工作负载配置文件。除了事务负载，我们现在还将考虑分析和混合配置文件（混合事务/分析处理）。此外，我们还将探讨如何以优化的方式协调多个动态竞争资源的并发DBMS实例。注释任务和数据对象的能力将用于进一步改进资源调度。查询翻译器应简化任务和注释的创建。基于任务的运行时系统的查询编译也有望提高效率。最终，对更好地利用现代众核系统的见解也将用于确定迁移路径，以减少破坏性但增量的方式改进现有DBMS和操作系统中使用的概念。

Emerging hardware platforms are characterized by large degrees of parallelism, complex memory hierarchies, and increasing hardware heterogeneity. Their theoretical peak data processing performance can only be unleashed if the different pieces of systems software collaborate much more closely and if their traditional dependencies and interfaces are redesigned.As a novel approach we have developed the key concepts and a prototype implementation of a bare-metal runtime system named MxKernel (first funding period of the project). MxKernel provides very lightweight resource management for database management system (DBMS) and operating system (OS), which both run as equal peers on top of the runtime. In MxKernel, heterogeneity and parallelism become first-class citizens and deep memory hierarchies are considered from the very beginning. Instead of a classical “thread” model, MxKernel provides a simpler control flow abstraction: MxTasks model closed units of work, for which MxKernel will guarantee the required execution semantics, such exclusive access to a specific object in memory. They can be a very elegant abstraction also for heterogeneity and resource sharing. Furthermore, MxTasks are annotated with meta data, such as code variants (to support heterogeneity), memory access behavior (to improve cache efficiency and support memory hierarchies), or dependencies between MxTasks (to improve scheduling and avoid synchronization cost). With precisely the required meta data available, MxKernel can provide a lightweight, yet highly efficient form of resource management, even across applications, operating system, and database.Based on the MxKernel prototype and our promising results, e.g. the ability to optimize memory usage by well-informed data prefetching, we are now going to explore and tap the full potential of the novel system software architecture in a more systematic way. For this purpose we will look into more realistic application scenarios and work load profiles. Besides transactional loads we will now also consider analytic and mixed profiles (hybrid transactional/analytic processing). Additionally, we will explore how multiple concurrent DBMS instances, which dynamically compete for resources, can be coordinated in an optimized way. The ability to annotate tasks and data objects shall be used for further improved resource scheduling. A query translator shall simplify the creation of tasks and annotations. Query compilation for the task-based runtime system can also be expected to improve the efficiency.Eventually, the gained insights on the better exploitation of modern manycore systems will also be used to identify migration paths to improve the concepts used in existing DBMS and OS in a less disruptive but incremental manner.