Improving Performance of Data Dumping with Lossy Compression for Scientific Simulation
Improving Performance of Data Dumping with Lossy Compression for Scientific Simulation
复制标题
通过有损压缩提高数据转储的性能以进行科学模拟
DOI：
10.1109/cluster.2019.8891037
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
F. Cappello
中科院分区：
文献类型：
--
作者：
Xin Liang;S. Di;Dingwen Tao;Sihuan Li;Bogdan Nicolae;Zizhong Chen;F. Cappello
由于当今高性能计算（HPC）科学模拟产生的数据不断增加，I/O性能正成为其执行的重要瓶颈。一个有效的错误控制的有损压缩器是一个有前途的解决方案，以显着减少数据写入时间的科学模拟运行在超级计算机上。在本文中，我们将探讨如何优化数据转储性能的科学模拟，利用误差有限的有损压缩技术。本文的贡献有三个方面。(1)提出了一种新的I/O性能分析模型，该模型能够有效地描述不同执行规模和数据大小下的I/O性能，并利用最小二乘法优化了数据转储性能的估计精度。(2)我们开发了一个自适应有损压缩框架，可以选择最适合的压缩机（两个领先的有损压缩机SZ和ZFP）与优化的参数设置方面的整体数据转储性能。(3)我们评估了我们的自适应有损压缩框架，在超级计算机上使用高达32 k个内核，并使用快速I/O系统和真实世界的科学模拟数据集。实验结果表明，我们的解决方案可以几乎总是导致数据转储性能的最佳水平，非常准确地选择最适合的有损压缩器和设置。数据转储性能在不同规模下可以提高高达27%。
Because of the ever-increasing data being produced by today’s high performance computing (HPC) scientific simulations, I/O performance is becoming a significant bottleneck for their executions. An efficient error-controlled lossy compressor is a promising solution to significantly reduce data writing time for scientific simulations running on supercomputers. In this paper, we explore how to optimize the data dumping performance for scientific simulation by leveraging error-bounded lossy compression techniques. The contributions of the paper are threefold. (1) We propose a novel I/O performance profiling model that can effectively represent the I/O performance with different execution scales and data sizes, and optimize the estimation accuracy of data dumping performance using least square method. (2) We develop an adaptive lossy compression framework that can select the bestfit compressor (between two leading lossy compressors SZ and ZFP) with optimized parameter settings with respect to overall data dumping performance. (3) We evaluate our adaptive lossy compression framework with up to 32k cores on a supercomputer facilitated with fast I/O systems and using real-world scientific simulation datasets. Experiments show that our solution can mostly always lead the data dumping performance to the optimal level with very accurate selection of the bestfit lossy compressor and settings. The data dumping performance can be improved by up to 27% at different scales.
DOI：
10.1109/ipdps.2018.00044
发表时间：
2018-05
期刊：
2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
影响因子：
--
作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao
通讯作者：
Tao Lu;Qing Liu;Xubin He;Huizhang Luo;E. Suchyta;J. Choi;N. Podhorszki;S. Klasky;M. Wolf;Tong Liu;Zhenbo Qiao