A methodology for evaluating the impact of data compression on climate simulation data
A methodology for evaluating the impact of data compression on climate simulation data
复制标题
DOI：
10.1145/2600212.2600217
复制
发表时间：
2014-06
期刊：
影响因子：
--
通讯作者：
A. Baker;Haiying Xu;J. Dennis;M. Levy;D. Nychka;S. Mickelson;Jim Edwards;M. Vertenstein;Al Wegener
中科院分区：
文献类型：
--
作者：
A. Baker;Haiying Xu;J. Dennis;M. Levy;D. Nychka;S. Mickelson;Jim Edwards;M. Vertenstein;Al Wegener
高分辨率气候模拟需要大量的计算资源，并且可以生成大量数据集。目前，保存这些模拟数据消耗了国家大气研究中心 (NCAR) 等机构的大量存储资源。历史数据生成趋势在经济上是不可持续的，并且存储资源已经开始限制科学目标。为了缓解这个问题，我们研究了对来自社区地球系统模型的气候模拟数据使用数据压缩技术。最终，为了说服气候科学家压缩他们的模拟数据，我们必须能够证明重建的数据揭示了与原始数据相同的平均气候，而本文是实现这一目标的第一步。为此，我们开发了一种验证气候数据的方法，并用它来评估几种压缩算法。我们发现，气候数据的多样性需要对变量进行单独处理，这样，重建的数据就可以落在系统的自然变化范围内，同时实现高达 5:1 的压缩率。
High-resolution climate simulations require tremendous computing resources and can generate massive datasets. At present, preserving the data from these simulations consumes vast storage resources at institutions such as the National Center for Atmospheric Research (NCAR). The historical data generation trends are economically unsustainable, and storage resources are already beginning to limit science objectives. To mitigate this problem, we investigate the use of data compression techniques on climate simulation data from the Community Earth System Model. Ultimately, to convince climate scientists to compress their simulation data, we must be able to demonstrate that the reconstructed data reveals the same mean climate as the original data, and this paper is a first step toward that goal. To that end, we develop an approach for verifying the climate data and use it to evaluate several compression algorithms. We find that the diversity of the climate data requires the individual treatment of variables, and, in doing so, the reconstructed data can fall within the natural variability of the system, while achieving compression rates of up to 5:1.