Ultrafast Error-Bounded Lossy Compression for Scientific Datasets
Ultrafast Error-Bounded Lossy Compression for Scientific Datasets
复制标题
科学数据集的超快误差限制有损压缩
DOI：
10.1145/3502181.3531473
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Cappello, Franck
中科院分区：
文献类型：
--
作者：
Yu, Xiaodong;Di, Sheng;Zhao, Kai;Tian, Jiannan;Tao, Dingwen;Liang, Xin;Cappello, Franck
当今科学的高性能计算应用和先进的仪器正在产生广泛领域的海量数据，这给数据传输和存储带来了严重的负担。误差界有损压缩不仅可以显著减少数据量，而且可以严格控制基于用户指定误差界的数据失真，因此在科学界得到了广泛的应用。然而，现有的有损压缩器不能提供超快的压缩速度，这是许多应用或用例(如内存压缩和在线仪器数据压缩)所高度要求的。在本文中，我们提出了一种新的超快误差界有损压缩算法，它在CPU和GPU上都能获得较高的压缩性能，并且具有较高的压缩比。主要贡献有三个方面。(1)提出了一种通用的误差界有损压缩框架--称为SZx--该框架通过其新颖的设计实现了超快的性能，该框架仅包含位和加/减等轻量级运算，同时仍保持较高的压缩比。(2)我们在CPU和GPU上实现了SZx，并根据它们的体系结构进行了性能优化。(3)我们在CPU和GPU上使用六个真实的生产级科学数据集进行了综合评估。实验表明，在CPU和GPU上，SZx在压缩和解压缩方面都比现有的第二快的误差界有损压缩算法(SZ或ZFP)快2~16倍。
Today's scientific high-performance computing applications and advanced instruments are producing vast volumes of data across a wide range of domains, which impose a serious burden on data transfer and storage. Error-bounded lossy compression has been developed and widely used in the scientific community because it not only can significantly reduce the data volumes but also can strictly control the data distortion based on the user-specified error bound. Existing lossy compressors, however, cannot offer ultrafast compression speed, which is highly demanded by numerous applications or use cases (such as in-memory compression and online instrument data compression). In this paper, we propose a novel ultrafast error-bounded lossy compressor that can obtain fairly high compression performance on both CPUs and GPUs and with reasonably high compression ratios. The key contributions are threefold. (1) We propose a generic error-bounded lossy compression framework---called SZx---that achieves ultrafast performance through its novel design comprising only lightweight operations such as bitwise and addition/subtraction operations, while still keeping a high compression ratio. (2) We implement SZx on both CPUs and GPUs and optimize the performance according to their architectures. (3) We perform a comprehensive evaluation with six real-world production-level scientific datasets on both CPUs and GPUs. Experiments show that SZx is 2~16x faster than the second-fastest existing error-bounded lossy compressor (either SZ or ZFP) on CPUs and GPUs, with respect to both compression and decompression.