Climate analytics on distributed exascale data archives - ExArch

气候科学对数据管理的要求正在迅速增长，因为气候模型在描述空间结构的精度和描述大量物理过程的完整性方面都在增长。对于气候模式相互比较项目5 (CMIP5)，正在构建一个分布式档案，以提供对预计超过10千兆字节的全球气候变化预测的访问。这些数据将保存在世界各地30个或更多的计算中心和数据档案中，但对用户来说，它将显示为由一个目录描述的单一档案。此外，数据的可用性将通过三步验证过程和发布所有数据的数字对象标识符（doi）来增强。对于许多用户来说，全球气候模式提供的空间分辨率（约150公里）是不够的：CORDEX项目将提供缩小到约10公里的数据。对气候影响的评估常常围绕极端情况和复杂的影响因素展开，需要存储大量数据。同时，关于模型最优配置的不确定性要求使用多个模型来探索每个场景。该项目将探索开发软件管理基础设施的挑战，该基础设施将扩展到多eb的气候数据档案，这些数据可能对本十年末的重大政策决策至关重要。支持归档数据和元数据的自动处理将是必不可少的。在短期目标中，将通过将策略应用于CORDEX项目数据来评估策略。

Climate science demands on data management are growing rapidly as climate models grow in the precision with which they depict spatial structures and in the completeness with which they describe a vast range of physical processes. For the Climate Model Inter-comparison Project 5 (CMIP5), a distributed archive is being constructed to provide access to what is expected to be in excess of 10 Peta-bytes of global climate change projections. The data will be held at 30 or more computing centres and data archives around the world, but for users it will appear as a single archive described by one catalogue. In addition, the usability of the data will be enhanced by a three-step validation process and the publication of Digital Object Identifiers (doi) for all the data. For many users the spatial resolution provided by the global climate models (around 150km) is inadequate: the CORDEX project will provide data scaled down to around 10km. Evaluation of climate impacts often revolves around extremes and complex impact factors, requiring high volumes of data to be stored. At the same time, uncertainty about the optimal configuration of the models imposes the requirement that each scenario be explored with multiple models. This project will explore the challenges of developing a software management infrastructure which will scale to the multi-exabyte archives of climate data which are likely to be crucial to major policy decisions in by the end of the decade. Support for automated processing of the archived data and metadata will be essential. In the short term goal, strategies will be evaluated by applying them to the CORDEX project data.