Accelerating Data Analytics Through Emerging Software-Hardware Mechanisms

本研究的目标是调查算法，设计和架构，使一个有效的实时事件数据分析平台，以支持多查询处理高容量和高频率的事件流。为了实现这些目标，我们计划利用现代硬件机制，如现场可编程门阵列（FPGA）和应用处理单元（APU）在我们的设计。我们努力实现线速多查询处理，利用前所未有的并行度和潜在的流水线，只有通过定制的，特定于应用程序和低层次的逻辑设计。此外，我们打算在我们的设计中使用新兴的图形处理单元（GPU）进行比较，首先评估性能，但也评估开发工作。对高效实时分析的需求是越来越多的数据管理问题场景（如业务分析、大数据处理和复杂事件处理）的一个组成部分。 所有这些场景的共同点是一组预定义的连续查询和传入数据的无界事件流，这些数据必须根据查询实时处理。**然而，使用FPGA增强数据分析计算的愿景有一些警告，使加速成为一项具有挑战性的任务。首先，与商品CPU相比，当前的FPGA仍然要慢得多。第二，加速的应用程序功能必须服从并行处理。第三，片上/片外数据速率必须跟上芯片处理速度，以通过保持定制的处理流水线忙碌来实现处理加速。最后，FPGA限制了设计人员的灵活性和应用程序的动态性，这两者在标准软件解决方案中几乎都不是问题。通过在我们的方法中应对这些挑战，我们提出了一个基于FPGA的实时分析平台，该平台支持在连续查询集合上对数据流进行线速处理。 我们计划沿着以下三个维度探索这个问题空间：（1）设计高吞吐量的自定义电路来实现关系代数运算符，（2）设计符合FPGA提供的功能的多查询优化技术，（3）设计软件到硬件的多查询处理技术，将一组查询映射到一个全局查询计划中，以便由我们的自定义电路进行处理。

The objectives of this research are to investigate algorithms, design, and architectures for enabling an efficient real-time event data analytics platform to support multi-query processing over high-volume and high-frequency event streams. To achieve these objectives, we plan to leverage modern hardware mechanisms, such as Field Programmable Gate Arrays (FPGAs) and Application Processing Units (APUs) in our design. We strive to achieve line-rate multi-query processing by exploiting unprecedented degrees of parallelism and potential for pipelining, only available through custom-built, application-specific and low-level logic design. Furthermore, we intend to compare against the use of emerging Graphical Processing Units (GPUs) in our design, evaluating above all performance, but also development effort.******The need for efficient real-time analytics is an integral part of a growing number of data management problem scenarios such as business analytics, big data processing, and complex event processing. Common among all these scenarios is a predefined set of continuous queries and an unbounded event stream of incoming data that must be processed against the queries in real-time.******However, the vision of enhancing data analytics computations with FPGAs has a few caveats that make acceleration a challenging undertaking. First, current FPGAs are still much slower compared to commodity CPUs. Second, the accelerated application functionality has to be amenable to parallel processing. Third, the on-/off-chip data rates must keep up with chip processing speeds to realize a processing speedup by keeping the custom-built processing pipeline busy. Finally, FPGAs restrict the designer's flexibility and the application's dynamism, both of which are hardly a concern in standard software solutions.******By meeting these challenges in our approach, we propose an FPGA-based real-time analytics platform that supports line-rate processing of data streams over a collection of continuous queries. We plan to explore this problem space along the following three dimensions: (1) Design high-throughput, custom circuits to implement the relational algebra operators, (2) design multi-query optimization techniques amenable to the features offered by FPGAs, (3) design software-to-hardware multi-query processing techniques that map a set of queries into a global query plan for processing by our custom circuits.