Significantly improving lossy compression quality based on an optimized hybrid prediction model
Significantly improving lossy compression quality based on an optimized hybrid prediction model
复制标题
基于优化的混合预测模型显着提高有损压缩质量
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
F. Cappello
中科院分区：
文献类型：
--
作者：
Xin Liang;S. Di;Sihuan Li;Dingwen Tao;Bogdan Nicolae;Zizhong Chen;F. Cappello
随着当今大规模科学模拟产生的数据量不断增加，误差有限的有损压缩技术变得至关重要：它们不仅可以显着减少数据大小，而且还可以为分析后保持高数据保真度。在本文中，我们设计了一个策略，以提高压缩质量显着的基础上优化，混合预测模型。我们的贡献是四方面的。(1)我们提出了一种新的，基于变换的预测器，并优化其压缩质量。(2)我们显着提高了数据拟合预测器的系数编码效率。(3)我们提出了一个自适应框架，可以为不同的数据集准确地选择最适合的预测。(4)我们评估我们的解决方案和现有的几个国家的最先进的有损压缩器上运行的实际应用程序与8,192个核心的超级计算机。实验结果表明，自适应压缩器与次优压缩器相比，压缩比提高了112~165%。并行I/O性能提高了约100%，因为数据大小显着减少。与原始I/O时间相比，使用我们的压缩器可将总I/O时间减少多达60倍。
With the ever-increasing volumes of data produced by today's large-scale scientific simulations, error-bounded lossy compression techniques have become critical: not only can they significantly reduce the data size but they also can retain high data fidelity for postanalysis. In this paper, we design a strategy to improve the compression quality significantly based on an optimized, hybrid prediction model. Our contribution is fourfold. (1) We propose a novel, transform-based predictor and optimize its compression quality. (2) We significantly improve the coefficient-encoding efficiency for the data-fitting predictor. (3) We propose an adaptive framework that can select the best-fit predictor accurately for different datasets. (4) We evaluate our solution and several existing state-of-the-art lossy compressors by running real-world applications on a supercomputer with 8,192 cores. Experiments show that our adaptive compressor can improve the compression ratio by 112~165% compared with the second-best compressor. The parallel I/O performance is improved by about 100% because of the significantly reduced data size. The total I/O time is reduced by up to 60X with our compressor compared with the original I/O time.
DOI：
10.1109/ipdps.2018.00044
发表时间：
2018-05
期刊：
2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
影响因子：
--
作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao
通讯作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao