Accelerating Data Analytics Through Emerging Software-Hardware Mechanisms

本研究的目标是研究支持高效实时事件数据分析平台的算法、设计和体系结构，以支持对大容量和高频事件流的多查询处理。为了实现这些目标，我们计划在我们的设计中利用现代硬件机制，例如现场可编程门阵列(FGA)和应用处理单元(APU)。我们努力通过开发前所未有的并行度和流水线的潜力来实现线速多查询处理，只有通过定制的、特定于应用程序的和低级别的逻辑设计才能做到这一点。此外，我们打算在我们的设计中与新兴图形处理单元(GPU)的使用进行比较，首先评估所有性能，但也评估开发工作。*对高效实时分析的需求是越来越多的数据管理问题场景(如业务分析、大数据处理和复杂事件处理)的组成部分。在所有这些场景中，常见的是预定义的连续查询集和传入数据流的无限事件流，这些数据流必须针对查询进行实时处理。*然而，使用FPGA增强数据分析计算的愿景有几个警告，这使得加速成为一项具有挑战性的任务。首先，与商用CPU相比，目前的现场可编程门阵列仍然慢得多。其次，加速的应用程序功能必须服从并行处理。第三，片上/片外数据速率必须跟上芯片处理速度，以通过保持定制处理流水线繁忙来实现处理加速。最后，FPGA限制了设计者的灵活性和应用程序的动态化，这两者在标准软件解决方案中几乎不是问题。*通过在我们的方法中应对这些挑战，我们提出了一个基于FPGA的实时分析平台，该平台支持对连续查询集合上的数据流进行线速处理。我们计划从以下三个方面探索这个问题：(1)设计高吞吐量的定制电路来实现关系代数算子，(2)设计符合FPGA提供的特征的多查询优化技术，(3)设计软件到硬件的多查询处理技术，将一组查询映射到全局查询计划中，供定制电路处理。

The objectives of this research are to investigate algorithms, design, and architectures for enabling an efficient real-time event data analytics platform to support multi-query processing over high-volume and high-frequency event streams. To achieve these objectives, we plan to leverage modern hardware mechanisms, such as Field Programmable Gate Arrays (FPGAs) and Application Processing Units (APUs) in our design. We strive to achieve line-rate multi-query processing by exploiting unprecedented degrees of parallelism and potential for pipelining, only available through custom-built, application-specific and low-level logic design. Furthermore, we intend to compare against the use of emerging Graphical Processing Units (GPUs) in our design, evaluating above all performance, but also development effort.******The need for efficient real-time analytics is an integral part of a growing number of data management problem scenarios such as business analytics, big data processing, and complex event processing. Common among all these scenarios is a predefined set of continuous queries and an unbounded event stream of incoming data that must be processed against the queries in real-time.******However, the vision of enhancing data analytics computations with FPGAs has a few caveats that make acceleration a challenging undertaking. First, current FPGAs are still much slower compared to commodity CPUs. Second, the accelerated application functionality has to be amenable to parallel processing. Third, the on-/off-chip data rates must keep up with chip processing speeds to realize a processing speedup by keeping the custom-built processing pipeline busy. Finally, FPGAs restrict the designer's flexibility and the application's dynamism, both of which are hardly a concern in standard software solutions.******By meeting these challenges in our approach, we propose an FPGA-based real-time analytics platform that supports line-rate processing of data streams over a collection of continuous queries. We plan to explore this problem space along the following three dimensions: (1) Design high-throughput, custom circuits to implement the relational algebra operators, (2) design multi-query optimization techniques amenable to the features offered by FPGAs, (3) design software-to-hardware multi-query processing techniques that map a set of queries into a global query plan for processing by our custom circuits.