FPGA Acceleration of Zstd Compression Algorithm
FPGA Acceleration of Zstd Compression Algorithm
复制标题
Zstd压缩算法的FPGA加速
DOI：
10.1109/ipdpsw52791.2021.00035
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Z. Al
中科院分区：
文献类型：
--
作者：
Jianyu Chen;M.A.F.M. Daverveldt;Z. Al
随着在各种应用领域（例如高频交易）中生成和存储的大数据量的持续增加，压缩技术对于降低通信带宽和存储容量的要求越来越重要。 ZSTANDARD（ZSTD）作为能够达到良好压缩比但速度高于可比算法的大数据集的重要压缩算法。在本文中，我们介绍了用于ZSTD的新硬件压缩内核的体系结构，该组件允许该算法用于实时压缩大数据流。此外，我们为流式传输高频交易数据的特定用例优化了所提出的体系结构。优化的内核是在Xilinx Alveo U200板上实现的。我们优化的实施使我们能够将十个内核块放在一个板上，该板能够达到约8.6GB/s的压缩吞吐量，压缩率约为23.6％。硬件实现是开源的，可在https://github.com/chenjianyunp/hardware-zstd-compression-unit上公开获得。
With the continued increase in the amount of big data generated and stored in various application domains, such as high-frequency trading, compression techniques are becoming ever more important to reduce the requirements on communication bandwidth and storage capacity. Zstandard (Zstd) is emerging as an important compression algorithm for big data sets capable of achieving a good compression ratio but with a higher speed than comparable algorithms. In this paper, we introduce the architecture of a new hardware compression kernel for Zstd that allows the algorithm to be used for real-time compression of big data streams. In addition, we optimize the proposed architecture for the specific use case of streaming high-frequency trading data. The optimized kernel is implemented on a Xilinx Alveo U200 board. Our optimized implementation allows us to fit ten kernel blocks on one board, which is able to achieve a compression throughput of about 8.6GB/s and compression ratio of about 23.6%. The hardware implementation is open source and publicly available at https://github.com/ChenJianyunp/Hardware-Zstd-Compression-Unit.