EarthCube Data Capabilities: A Cloud-Native Data Repository for the Geoscience Community

EarthCube 数据功能：地球科学社区的云原生数据存储库

• 批准号: 2026932
• 负责人: Ryan Abernathey
• 金额: $ 159.99万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2026932&HistoricalAwards=false
• 关键词: EarthCube; Data; Capabilities; Cloud; Native

The Pangeo Project’s overall goal is to make scientists more productive by making it easier, more efficient, and more cost effective to work with very large datasets. This project focuses on using cloud computing to transform scientists’ relationship with data. While most traditional data repositories provide download services, this project will create a “cloud native data repository” to instead allow scientists to do computing in the cloud, where the data are already present and ready for analysis. Software tools and infrastructure will be developed to enable scientists and data providers to extract data from its original location and store it in the cloud in an optimized format. Partnerships with industry will help ensure that this innovative technological approach helps enhance US economic competitiveness.This project will build a new type of data and computing facility for the geoscience community using cloud technology. Central to this will be a repository of analysis-ready data stored in cloud-native formats such as Zarr. The Pangeo cloud data repository will enable scientists to create analysis-ready copies of existing datasets while fully tracing their provenance to the original source repository. This new virtual facility will transform how scientists work and collaborate, leading to massive leaps in productivity for the entire field. The infrastructure will be built using modular, reusable, sustainable individual software components, including Pangeo Forge–a tool for automating the production of analysis-ready data–and a cloud-native data catalog and accompanying website. Partnering with existing data providers and science communities in the areas of ocean observatories, coupled climate modeling, satellite observations, and weather reanalysis will ensure that the repository is maximally useful to the research community. This award by the Directorate for Geosciences is jointly supported by the Office of Advanced Cyberinfrastructure.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Pangeo项目的总体目标是通过使科学家更容易、更高效、更经济地处理大型数据集来提高科学家的工作效率。该项目的重点是使用云计算来改变科学家与数据的关系。虽然大多数传统的数据存储库提供下载服务，但该项目将创建一个“云原生数据存储库”，以允许科学家在云中进行计算，其中数据已经存在并准备好进行分析。将开发软件工具和基础设施，使科学家和数据提供商能够从原始位置提取数据，并以优化格式将其存储在云中。与工业界的合作将有助于确保这种创新的技术方法有助于提高美国的经济竞争力。该项目将利用云技术为地球科学界建立一种新型的数据和计算设施。其核心将是以Zarr等云原生格式存储的分析就绪数据存储库。Pangeo云数据存储库将使科学家能够创建现有数据集的分析就绪副本，同时完全跟踪其来源到原始源存储库。这个新的虚拟设施将改变科学家的工作和协作方式，从而使整个领域的生产力实现巨大飞跃。该基础设施将使用模块化、可重用、可持续的单个软件组件构建，包括Pangeo Forge（一种用于自动生成分析就绪数据的工具）和云原生数据目录及配套网站。与海洋观测站、耦合气候建模、卫星观测和天气再分析等领域的现有数据提供者和科学界合作，将确保储存库对研究界发挥最大的作用。该奖项由地球科学理事会颁发，并得到高级网络基础设施办公室的共同支持。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Pangeo Forge: Crowdsourcing Analysis-Ready, Cloud Optimized Data Production

Pangeo Forge：众包分析就绪、云优化的数据生产

• DOI: 10.3389/fclim.2021.782909
• 发表时间: 2022
• 期刊: Frontiers in Climate
• 作者: Stern, Charles;Abernathey, Ryan;Hamman, Joseph;Wegener, Rachel;Lepore, Chiara;Harkins, Sean;Merose, Alexander
• 通讯作者: Merose, Alexander

Future Global Convective Environments in CMIP6 Models

• DOI: 10.1029/2021ef002277
• 发表时间: 2021-12-01
• 期刊: EARTHS FUTURE
• 影响因子: 8.2
• 作者: Lepore, Chiara;Abernathey, Ryan;Tippett, Michael K.
• 通讯作者: Tippett, Michael K.

Cloud-Native Repositories for Big Scientific Data

• DOI: 10.1109/mcse.2021.3059437
• 发表时间: 2021-03-01
• 期刊: COMPUTING IN SCIENCE & ENGINEERING
• 影响因子: 2.1
• 作者: Abernathey, Ryan P.;Blackmon-Luca, Charles C.;Signell, Richard P.
• 通讯作者: Signell, Richard P.