MxKernel: A Bare-Metal Runtime System for Database Operations on Heterogeneous Many-Core Hardware

新兴的硬件平台的特点是高度并行性、复杂的内存层次结构和日益增长的硬件异构性。只有当不同的系统软件更紧密地协作，并重新设计它们的传统依赖项和接口时，它们的理论峰值数据处理性能才能得到释放。作为一种新的方法，我们开发了一个名为MxKernel(项目的第一个资助期)的裸机运行时系统的关键概念和原型实现。MxKernel为数据库管理系统(DBMS)和操作系统(OS)提供了非常轻量级的资源管理，它们都作为运行时之上的对等设备运行。在MxKernel中，异构性和并行性成为一等公民，并从一开始就考虑了深层存储层次结构。与经典的“线程”模型不同，MxKernel提供了一个更简单的控制流抽象：MxTasksModelOf Work Units，MxKernel将保证所需的执行语义，比如对内存中特定对象的独占访问。对于异构性和资源共享，它们也可以是一个非常优雅的抽象。此外，MxTask用元数据进行注释，例如代码变体(以支持异构性)、内存访问行为(以提高缓存效率并支持内存层次结构)或MxTaskTM之间的依赖关系(以改进调度并避免同步成本)。通过提供所需的元数据，MxKernel可以提供一种轻量级但高效的资源管理形式，甚至可以跨应用程序、操作系统和数据库进行管理。基于MxKernel原型和我们令人振奋的成果，例如通过充分了解数据预取来优化内存使用的能力，我们现在将以更系统的方式探索和挖掘这种新型系统软件体系结构的全部潜力。为此，我们将研究更实际的应用场景和工作负载配置文件。除了事务负载，我们现在还将考虑分析和混合配置文件(混合事务/分析处理)。此外，我们还将探讨如何以优化的方式协调动态竞争资源的多个并发DBMS实例。注释任务和数据对象的能力应用于进一步改进资源调度。查询翻译器应简化任务和注释的创建。基于任务的运行时系统的查询编译也有望提高效率。最后，关于更好地开发现代多核系统的见解也将被用来确定迁移路径，以更少的干扰和增量的方式改进现有数据库管理系统和操作系统中使用的概念。

Emerging hardware platforms are characterized by large degrees of parallelism, complex memory hierarchies, and increasing hardware heterogeneity. Their theoretical peak data processing performance can only be unleashed if the different pieces of systems software collaborate much more closely and if their traditional dependencies and interfaces are redesigned.As a novel approach we have developed the key concepts and a prototype implementation of a bare-metal runtime system named MxKernel (first funding period of the project). MxKernel provides very lightweight resource management for database management system (DBMS) and operating system (OS), which both run as equal peers on top of the runtime. In MxKernel, heterogeneity and parallelism become first-class citizens and deep memory hierarchies are considered from the very beginning. Instead of a classical “thread” model, MxKernel provides a simpler control flow abstraction: MxTasks model closed units of work, for which MxKernel will guarantee the required execution semantics, such exclusive access to a specific object in memory. They can be a very elegant abstraction also for heterogeneity and resource sharing. Furthermore, MxTasks are annotated with meta data, such as code variants (to support heterogeneity), memory access behavior (to improve cache efficiency and support memory hierarchies), or dependencies between MxTasks (to improve scheduling and avoid synchronization cost). With precisely the required meta data available, MxKernel can provide a lightweight, yet highly efficient form of resource management, even across applications, operating system, and database.Based on the MxKernel prototype and our promising results, e.g. the ability to optimize memory usage by well-informed data prefetching, we are now going to explore and tap the full potential of the novel system software architecture in a more systematic way. For this purpose we will look into more realistic application scenarios and work load profiles. Besides transactional loads we will now also consider analytic and mixed profiles (hybrid transactional/analytic processing). Additionally, we will explore how multiple concurrent DBMS instances, which dynamically compete for resources, can be coordinated in an optimized way. The ability to annotate tasks and data objects shall be used for further improved resource scheduling. A query translator shall simplify the creation of tasks and annotations. Query compilation for the task-based runtime system can also be expected to improve the efficiency.Eventually, the gained insights on the better exploitation of modern manycore systems will also be used to identify migration paths to improve the concepts used in existing DBMS and OS in a less disruptive but incremental manner.