ZCluster: A Zynq-based Hadoop cluster
ZCluster: A Zynq-based Hadoop cluster
复制标题
ZCluster：基于 Zynq 的 Hadoop 集群
DOI：
10.1109/fpt.2013.6718411
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
P. Chow
中科院分区：
文献类型：
--
作者：
Zhongduo Lin;P. Chow
基于 ARM 的服务器因其低功耗而越来越受到大数据处理的关注。然而，由于与传统服务器中使用的 CPU 相比，它们的处理能力较差，因此不适合计算密集型任务。本文介绍了我们将 FPGA 的处理能力与 Xilinx Zynq SoC 内的 ARM 处理器集成的早期努力。构建了基于 Zynq 的八从机 Hadoop 集群，并实现了针对标准 FIR 滤波器的定制硬件加速器，以验证硬件加速的有效性。 Xillybus用于ARM处理器和FPGA结构之间的通信，实现了103MB/s的带宽。事实证明，Hadoop 集群可以随着不同的输入大小和从属数量进行线性扩展。总体而言，与单个 ARM 处理器上的本机纯软件实现相比，该集群实现了 3.3 倍的加速，与不带硬件加速器的基于 ARM 的集群相比，性能提高了约 20%。
ARM-based servers are garnering increasing interest in big data processing for their low power consumption. However, they are ill-suited for compute-intensive tasks due to their poor processing capability compared to the CPUs used in a traditional server. This paper describes our early efforts to integrate the processing power of the FPGA with the ARM processor inside the Xilinx Zynq SoC. An eight-slave Zynq-based Hadoop cluster is built and a customized hardware accelerator for a standard FIR filter is implemented to demonstrate the effectiveness of hardware acceleration. The Xillybus is used for communication between the ARM processor and the FPGA fabric, achieving a bandwidth of 103MB/s. The Hadoop cluster is proved to be linearly scalable with different input sizes and numbers of slaves. Overall, the cluster achieves a 3.3-fold speedup compared to a native pure software implementation on a single ARM processor and about a 20% improvement compared to an ARM-based cluster without hardware accelerators.