ADAMANT: Adaptive Data Management in Evolving Heterogeneous Hardware/Software Systems

与纯粹的基于CPU的系统相比，由CPU、GPU和FPGA组成的异类系统架构为数据库系统提供了多种优化可能性。然而，已经表明，仅仅将现有的软件概念一对一地映射到非冯-诺伊曼硬件体系结构(如现场可编程门阵列)以充分发挥其优化潜力是不够的。相反，新的处理能力需要设计新的处理概念，这必须在查询处理的规划级别进行考虑。通过考虑我们的即插即用系统架构中的设备特定功能，已经在第一个项目阶段开发了一个基本的处理概念。事实上，需要更先进的概念来实现对硬件体系结构功能的最佳利用。虽然在映射到图形处理器和现场可编程门阵列的单个运算符的级别上实现了显著的加速，但在完整查询级别上的性能提升并不令人满意。因此，我们提出了第二阶段的假设，即标准的查询映射方法在单个操作符的级别上考虑查询，不足以探索异构系统体系结构的扩展处理特征，我们将通过研究新的处理和查询映射方法来解决这一缺陷，这些方法对常用的操作符粒度级别提出了质疑。因此，我们将提供封装了比标准数据库操作符更大功能的处理实体，并且可能跨越多个硬件设备。因此，处理实体本质上是异类的，并且结合了各个设备的特定特征。因此，我们的异类系统架构支持在传统数据库系统中不可用或无法有效实施的数据库操作和功能。为了探索这一扩展的特征集，我们确定了三个应用领域，它们对传统数据库系统仍然具有挑战性，我们认为它们将从异类系统体系结构中受益匪浅：大容量数据馈送、近似查询处理和动态多查询处理。大容量数据馈送的基于流的性质要求一种硬件体系结构，在这种体系结构中，处理可以在不需要预先存储数据的情况下即时完成。因此，现场可编程门阵列是处理大容量数据馈送应用的一种很有前途的硬件平台。此外，FPGA和GPU都是进行近似查询处理的良好平台，因为它们支持近似算法和受硬件影响的采样技术。从系统管理的角度来看，动态多查询处理是非常具有挑战性的，因为对于一个工作负载执行良好的查询计划对于不同的工作负载可能效率低下。在这里，异类系统的多级并行性为处理繁重的工作负载提供了更好的机会。

Heterogeneous system architectures consisting of CPUs, GPUs and FPGAs offer a variety of optimization possibilities for database systems compared to pure CPU-based systems. However, it has been shown that it is not sufficient to just map existing software concepts one-to-one to non von-Neumann hardware architectures such as FPGAs to fully exploit their optimization potential. Rather, new processing capabilities require the design of novel processing concepts, which have to be considered at the planning level of query processing. A basic processing concept has already been developed in the first project phase by considering device-specific features in our plug’n’play system architecture. In fact, more advanced concepts are required to achieve an optimal exploitation of the capabilities of the hardware architectures. While significant speed-ups were achieved on the level of individual operators mapped to GPUs and FPGAs, the performance gain at the level of complete queries was unsatisfying. Hence, we derived the hypothesis for the second project phase that standard query-mapping approaches with their consideration of queries on the level of individual operators is not sufficient to explore the extended processing features of heterogeneous system architectures.We will address this shortcoming by researching new processing and query mapping methods for heterogeneous systems, which question the commonly used granularity level of operators. Therefore, we will provide processing entities that encapsulate a greater functionality than standard database operators and may span multiple hardware devices. Thus, processing entities are intrinsically heterogeneous and combine the specific features of individual devices. As a result, our heterogeneous system architecture enables database operations and features that are not available or cannot be implemented efficiently in classical database systems. To explore this extended feature set, we have identified three application domains that are still challenging for classical database systems and for which we assume that they will benefit greatly from heterogeneous system architectures: High-volume data feeds, approximate query processing and dynamic multi-query processing. The stream-based nature of high-volume data feeds asks for a hardware architecture where processing can be done on the fly without the need to store data beforehand. Hence, FPGAs are a promising hardware platform for processing high-volume data feed applications. Furthermore, FPGAs as well as GPUs are good platforms for approximate query processing, as they allow for approximate arithmetics and hardware-influenced sampling techniques. Dynamic multi-query processing is very challenging from the system management point of view, as query plans that have performed well for one workload can be inefficient for a different workload. Here, the multi-level parallelism of heterogeneous systems offers better opportunities to handle heavy workloads.