SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors
SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors
复制标题
SZ3：用于组合基于预测的误差有限有损压缩器的模块化框架
DOI：
10.1109/tbdata.2022.3201176
复制
发表时间：
2023
影响因子：
7.2
通讯作者：
Tao, Dingwen
中科院分区：
文献类型：
--
作者：
Liang, Xin;Zhao, Kai;Di, Sheng;Li, Sihuan;Underwood, Robert;Gok, Ali M.;Tian, Jiannan;Deng, Junjing;Calhoun, Jon C.;Tao, Dingwen
今天的科学模拟需要显著减少数据量，因为它们产生的数据量非常大，并且I/O带宽和存储空间有限。错误有界有损压缩被认为是解决上述问题最有效的方法之一。但在实际应用中，由于不同数据集的特点不同，用户对压缩质量和性能的要求也不同，因此往往需要对最适合的压缩方法进行定制或优化。在本文中，我们用一个新的模块化、可组合的压缩框架SZ3来解决这个问题。我们的贡献是四倍的。(1)我们开发了SZ3，它对基于预测的压缩框架进行了创新的模块化抽象，这样可以很容易地插入压缩模块，根据数据特征和用户需求创建新的压缩器。(2)我们通过SZ3为GAMESS数据创建了一个新的压缩管道，这比最先进的压缩机显著提高了压缩比。(3)我们用SZ3开发了一种自适应压缩管道，以最小的努力对APS数据进行压缩，在所有现有的错误有界有损压缩器中，对于任何比特率，它的率失真是最好的。(4)我们将SZ3的可持续性与领先的基于误差有界预测的压缩器进行了比较，然后通过对来自多个学科的不同科学数据集的几种压缩管道进行整合和评估，证明了多种管道的必要性。实验表明，SZ3在压缩器集成方面的开销非常有限，与现有最佳方法相比，在相同数据失真的情况下，我们定制的压缩管道可将压缩比提高20%。
Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Error-bounded lossy compression has been considered one of the most effective solutions to the above problem. In practice, however, the best-fit compression method often needs to be customized or optimized in particular because of diverse characteristics in different datasets and various user requirements on the compression quality and performance. In this paper, we address this issue with a novel modular, composable compression framework named SZ3. Our contributions are four-folds. (1) We develop SZ3 which features an innovative modular abstraction for the prediction-based compression framework, such that compression modules can be plugged in easily to create new compressors based on characteristics of data and user requirements. (2) We create a new compression pipeline by SZ3 for GAMESS data, which significantly improves the compression ratios over state-of-the-art compressors. (3) We develop an adaptive compression pipeline by SZ3 for APS data with minimal efforts, which leads to the best rate-distortion among all existing error-bounded lossy compressors for any bit-rate. (4) We compare the sustainability of SZ3 with leading error-bounded prediction-based compressors, and then demonstrate the necessity of diverse pipelines by integrating and evaluating several compression pipelines on diverse scientific datasets from multiple disciplines. Experiments show that SZ3 incurs very limited overhead in compressor integration and our customized compression pipelines lead to up to 20% improvement in compression ratios under the same data distortion, when compared with the best existing approach.
登录
查看更多内容
保持对缩减的信任：保持有损压缩感兴趣数量的准确性
DOI：
10.1007/978-3-030-96498-6_2
发表时间：
2021
期刊：
SC14: International Conference for High Performance Computing, Networking, Storage and Analysis
影响因子：
--
作者：
Qian Gong;Xin Liang;Ben Whitney;J. Choi;Jieyang Chen;Lipeng Wan;S. Ethier;S. Ku;R. Churchill;Choong;M. Ainsworth;O. Tugluk;T. Munson;D. Pugmire;Rick Archibald;S. Klasky
通讯作者：
S. Klasky
基于优化的混合预测模型显着提高有损压缩质量
DOI：
--
发表时间：
2019
期刊：
International Conference on Software Composition
影响因子：
--
作者：
Xin Liang;S. Di;Sihuan Li;Dingwen Tao;Bogdan Nicolae;Zizhong Chen;F. Cappello
通讯作者：
F. Cappello
SZx：用于科学数据集的超快速误差有界有损压缩器
DOI：
--
发表时间：
2022
期刊：
arXiv.org
影响因子：
--
作者：
Xiaodong Yu;S. Di;Kai Zhao;Jiannan Tian;Dingwen Tao;Xin Liang;F. Cappello
通讯作者：
F. Cappello
OptZConfig：有损压缩配置的高效并行优化
影响因子：
5.3
作者：
Underwood, Robert;Calhoun, Jon C;Di, Sheng;Apon, Amy;Cappello, Franck
通讯作者：
Cappello, Franck
DOI：
10.1109/cluster48925.2021.00034
发表时间：
2021-05
期刊：
2021 IEEE International Conference on Cluster Computing (CLUSTER)
影响因子：
--
作者：
Jinyang Liu;S. Di;Kai Zhao;Sian Jin;Dingwen Tao;Xin Liang;Zizhong Chen;F. Cappello
通讯作者：
Jinyang Liu;S. Di;Kai Zhao;Sian Jin;Dingwen Tao;Xin Liang;Zizhong Chen;F. Cappello