In-depth exploration of single-snapshot lossy compression techniques for N-body simulations
In-depth exploration of single-snapshot lossy compression techniques for N-body simulations
复制标题
深入探索 N 体模拟的单快照有损压缩技术
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
F. Cappello
中科院分区：
文献类型：
--
作者：
Dingwen Tao;S. Di;Zizhong Chen;F. Cappello
允许用户控制数据丢失的原位有损压缩可以显著减少I/O负担。对于一次只能压缩一个快照的大规模N体模拟，由于粒子数据的空间相干性相当低，有损压缩比非常有限。在这项工作中，我们评估国家的最先进的单快照有损压缩技术的两个常见的N体模拟模型：宇宙学和分子动力学。我们设计了一系列新的优化技术的基础上的两个代表性的现实世界的N体仿真代码。对于分子动力学模拟，我们提出了三种压缩模式（即，最佳速度、最佳折衷、最佳压缩模式），其可以细化压缩率（也称为，速度/吞吐量）和比率。对于宇宙学模拟，我们确定我们的改进SZ是最好的有损压缩器的压缩比和速率。在压缩比相当的情况下，其压缩比比第二好的压缩机高出11%。在阿贡的Blues超级计算机上进行的多达1024个核的实验表明，与直接将数据写入并行文件系统相比，我们提出的有损压缩方法可以减少80%的I/O时间，并且比第二好的解决方案的性能高出60%。此外，我们提出的有损压缩方法具有最好的率失真与合理的压缩误差的测试N体模拟数据相比，国家的最先进的压缩机。
In situ lossy compression allowing user-controlled data loss can significantly reduce the I/O burden. For large-scale N-body simulations where only one snapshot can be compressed at a time, the lossy compression ratio is very limited because of the fairly low spatial coherence of the particle data. In this work, we assess the state-of-the-art single-snapshot lossy compression techniques of two common N-body simulation models: cosmology and molecular dynamics. We design a series of novel optimization techniques based on the two representative real-world N-body simulation codes. For molecular dynamics simulation, we propose three compression modes (i.e., best speed, best tradeoff, best compression mode) that can refine the tradeoff between the compression rate (a.k.a., speed/throughput) and ratio. For cosmology simulation, we identify that our improved SZ is the best lossy compressor with respect to both compression ratio and rate. Its compression ratio is higher than the second-best compressor by 11% with comparable compression rate. Experiments with up to 1024 cores on the Blues supercomputer at Argonne show that our proposed lossy compression method can reduce I/O time by 80% compared with writing data directly to a parallel file system and outperforms the second-best solution by 60%. Moreover, our proposed lossy compression methods have the best rate-distortion with reasonable compression errors on the tested N-body simulation data compared with state-of-the-art compressors.