Adaptive Configuration of In Situ Lossy Compression for Cosmology Simulations via Fine-Grained Rate-Quality Modeling
Adaptive Configuration of In Situ Lossy Compression for Cosmology Simulations via Fine-Grained Rate-Quality Modeling
复制标题
通过细粒度速率-质量建模进行宇宙学模拟的原位有损压缩的自适应配置
DOI：
10.1145/3431379.3460653
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Ahrens, James
中科院分区：
文献类型：
--
作者：
Jin, Sian;Pulido, Jesus;Grosset, Pascal;Tian, Jiannan;Tao, Dingwen;Ahrens, James
极端尺度的宇宙学模拟已经被当今的研究人员和科学家广泛应用于领导超级计算机。新一代错误限制有损压缩器已被用于工作流中，以降低存储要求并最大限度地减少吞吐量限制的影响，同时保存高保真数据的大型快照以供事后分析。在本文中，我们建议自适应地提供压缩配置计算分区的宇宙学模拟与新设计的后分析感知率质量建模。贡献有四个方面：（1）我们提出了一种新的自适应方法来为不同的分区选择可行的误差范围，显示了为每个分区分别自适应配置有损压缩的可能性和效率。(2)我们建立模型来估计由于有损压缩而导致的后分析结果的总体损失，并根据每个分区的属性来估计压缩比。(3)我们开发了一个有效的优化准则，以确定最适合的配置的误差界组合，以最大限度地提高压缩比下可接受的分析后的质量损失。(4)我们的方法引入了可以忽略不计的开销的特征提取和错误约束优化每个分区，使后分析感知原位有损压缩宇宙学模拟。实验表明，我们提出的模型是高度准确和可靠的。我们的细粒度自适应配置方法在测试数据集上提高了高达73%的压缩率，具有相同的分析后失真，性能开销仅为1%。
Extreme-scale cosmological simulations have been widely used by today's researchers and scientists on leadership supercomputers. A new generation of error-bounded lossy compressors has been used in workflows to reduce storage requirements and minimize the impact of throughput limitations while saving large snapshots of high-fidelity data for post-hoc analysis. In this paper, we propose to adaptively provide compression configurations to compute partitions of cosmological simulations with newly designed post-analysis aware rate-quality modeling. The contribution is fourfold: (1) We propose a novel adaptive approach to select feasible error bounds for different partitions, showing the possibility and efficiency of adaptively configuring lossy compression for each partition individually. (2) We build models to estimate the overall loss of post-analysis result due to lossy compression and to estimate compression ratio, based on the property of each partition. (3) We develop an efficient optimization guideline to determine the best-fit configuration of error bounds combination in order to maximize the compression ratio under acceptable post-analysis quality loss. (4) Our approach introduces negligible overheads for feature extraction and error-bound optimization for each partition, enabling post-analysis-aware in situ lossy compression for cosmological simulations. Experiments show that our proposed models are highly accurate and reliable. Our fine-grained adaptive configuration approach improves the compression ratio of up to 73% on the tested datasets with the same post-analysis distortion with only 1% performance overhead.
登录
查看更多内容
前瞻：对于数据缩减至关重要的分析
DOI：
--
发表时间：
2020
期刊：
International Conference for High Performance Computing, Networking, Storage and Analysis
影响因子：
--
作者：
Pascal Grosset;C. Biwer;Jesus Pulido;A. Mohan;Ayan Biswas;J. Patchett;Terece L. Turton;D. Rogers;D. Livescu;J. Ahrens
通讯作者：
J. Ahrens
宇宙学模拟数据集有损压缩模式匹配技术的探索
DOI：
10.1007/978-3-319-67630-2_4
发表时间：
2017
期刊：
2018 IEEE International Conference on Cluster Computing (CLUSTER)
影响因子：
--
作者：
Dingwen Tao;S. Di;Zizhong Chen;F. Cappello
通讯作者：
F. Cappello
生产领先级存储系统中用户感知 I/O 性能的综合测量和分析
DOI：
--
发表时间：
2017
期刊：
IEEE International Conference on Distributed Computing Systems
影响因子：
--
作者：
Lipeng Wan;M. Wolf;Feiyi Wang;J. Choi;G. Ostrouchov;S. Klasky
通讯作者：
S. Klasky
DOI：
10.1109/ipdps.2018.00044
发表时间：
2018-05
期刊：
2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
影响因子：
--
作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao
通讯作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao
cuSZ：一种基于 GPU 的高效科学数据误差有限有损压缩框架
DOI：
--
发表时间：
2020
期刊：
Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques
影响因子：
--
作者：
Jiannan Tian, Sheng Di
通讯作者：
Jiannan Tian, Sheng Di