Exploring Best Lossy Compression Strategy By Combining SZ with Spatiotemporal Decimation
Exploring Best Lossy Compression Strategy By Combining SZ with Spatiotemporal Decimation
复制标题
将 SZ 与时空抽取相结合探索最佳有损压缩策略
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
F. Cappello
中科院分区：
文献类型：
--
作者：
Xin Liang;S. Di;Sihuan Li;Dingwen Tao;Zizhong Chen;F. Cappello
—在当今超大规模的科学模拟中，会产生大量数据，导致并行文件系统无法容纳这些数据，或者由于 I/O 带宽有限，导致数据写入/读取性能相当低。在过去的十年中，已经开发了许多基于快照（或基于空间）的有损压缩器，其中大多数依赖于空间中数据的平滑性。然而，随着时间的推移，模拟数据在空间上可能会变得越来越复杂，从而导致压缩比显着降低。在本文中，我们通过利用 SZ 压缩模型下的时空抽取，提出了一种新颖的混合有损压缩方法。贡献是双重的。 (1)在仿真过程中，我们在空间维度和时间维度上探索了抽取方法与SZ有损压缩模型相结合的几种策略。 （2）我们基于多个领域的几个典型的现实世界模拟，研究了不同需求的最佳组合策略。实验表明，基于空间的 SZ 或基于时间的 SZ 都会产生最佳的速率失真。抽取方法具有非常高的压缩率和低率失真，并且 SZ 与时间抽取相结合是一个很好的权衡。
—In today’s extreme-scale scientific simulations, vast volumes of data are being produced such that the data cannot be accommodated by the parallel file system or the data writing/reading performance will be fairly low because of limited I/O bandwidth. In the past decade, many snapshot-based (or space-based) lossy compressors have been developed, most of which rely on the smoothness of the data in space. However, the simulation data may get more and more complicated in space over time steps, such that the compression ratios decrease significantly. In this paper, we propose a novel, hybrid lossy compression method by leveraging spatiotemporal decimation under the SZ compression model. The contribution is twofold. (1) We explore several strategies of combining the decimation method with the SZ lossy compression model in both the space dimension and time dimension during the simulation. (2) We investigate the best-fit combined strategy upon different demands based on a couple of typical real-world simulations with multiple fields. Experiments show that either the space-based SZ or time-based SZ leads to the best rate distortion. Decimation methods have very high compression rate with low rate distortion though, and SZ combined with temporal decimation is a good tradeoff.
DOI：
10.1109/ipdps.2018.00044
发表时间：
2018-05
期刊：
2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
影响因子：
--
作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao
通讯作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao