EarthCube Building Blocks: Collaborative Proposal: Cloud-Hosted Real-time Data Services for the Geosciences (CHORDS)

实时科学数据的重要性不断增加，特别是在必须快速做出明智决策的关键任务场景中。此外，实时数据分布的进步正在导致时空中许多新的瞬态现象被观察到。然而目前，实时决策在许多情况下是不可行的，需要流式科学数据与复杂模型相结合。虽然 EarthCube 将为传播数据源提供前所未有的框架，但实时数据的使用提出了一系列必须考虑的额外复杂挑战。该项目是一个试点项目，旨在展示围绕实时数据收集和使用协调地球科学的重要性。这项工作将展示云托管的地球科学实时数据服务 (CHORDS)。 CHORDS 背后的愿景是提供一个实时数据管理基础设施，该基础设施将： a) 提供一个通过互联网存档、导航和分发实时数据流的系统； b)易于部署和配置； c) 在云基础设施上运行； d) 使用基于 RESTful 协议构建的事务（即通过 URL）； e) 采用符合标准的数据和元数据格式，从而简化用户体验； f) 免费和开源。例如，从观察平台数据中得出的科学是部署前多年规划的结果，部署本身花费了数百万美元，希望获得所需的数据集。许多地球科学实验往往资源有限。在这种情况下，保证资源的最佳分配至关重要，以确保重要事件不会被忽视。在地球科学领域，在现场活动后分析数据的情况并不少见，只是为了检测传感器故障、校准偏移或异常行为而为时已晚。实时数据将通过自动检测故障和异常来实现有限实验资源的最佳分配。 CHORDS将提供一个框架，实现各种实时地球科学数据源的自适应采样、发现和融合，从而促进地球科学实验的新开展。用例将通过显示传统实验由于无法访问实时数据而错过感兴趣的事件来说明这一点。专注于定义需求、设计和规范的初始工作。开始将一小部分地球科学数据流摄取到云中构建的原型 CHORDS 结构中。参与加强通过 CHORDS 摄取的实时数据与正在开发的其他 EarthCube Building Block 系统集成的活动。他们将重点关注水文传感器数据、雷达数据流、NCAR 低层大气观测设施以及地球和海洋社区外展方面的一些初步测试案例。

The importance of real-time scientific data is ever increasing, particularly in mission critical scenarios, where informed decisions must be made rapidly. Furthermore, advances in the distribution of real-time data are leading many new transient phenomena in space-time to be observed. Presently however, realtime decision-making is infeasible in many cases that require streaming scientific data to be coupled with complex models. While EarthCube will provide an unprecedented framework for disseminating data sources, the use of real-time data raises an additional set of complex challenges that must be considered. This project is a pilot to demonstrate the importance of coodinating the geosciences around their real-time data gathering and use. The work will demonstrate a Cloud-Hosted Real-time Data Services for the Geosciences (CHORDS)The vision behind CHORDS is to provide a real-time data management infrastructure that will: a)Provide a system to archive, navigate and distribute real-time data streams via the Internet; b)Be easily deployed and configured; c)Run on cloud infrastructure; d)Use transactions built on RESTful protocols (i.e. via URLs); e)Employ data and metadata formats that adhere to standards, which simplify the user experience; f)Be free and open source.Science derived from observing platform data is the result of years of planning before a deployment, for example, and millions of dollars are spent on the deployment itself in hopes of obtaining the desired dataset. Many geo-scientific experiments are often resource constrained. In such cases it is vital to guarantee the optimal allocation of resources to ensure that important events do not go unobserved. In geo-scientific domains it is not uncommon to analyze data after a field campaigns, only to detect sensor faults, calibration offsets or anomalous behaviors when it is already too late. Real-time data will enable the optimal allocation of constrained experimental resources by automating the detection of faults and anomalies. CHORDS will provide a framework to enable adaptive sampling, discovery and fusion of various real-time geoscientific data sources, thus facilitating a new means by which geoscience experiments are carried out. The use cases will illustrate this by showing traditional experiments would have missed events of interest due to lack of access to real-time data. Focus on the initial work of defining requirements, design and specifications. Begin to ingest a small subset of geosciences data streams into a prototype CHORDS structure built in the cloud. Participate in activities that strengthen the integration of real-time data being ingested via CHORDS into other EarthCube Building Block systems that are under development. They will focus on some initial test cases, in hydrology sensor data, radar data streams, the NCAR Lower Atmosphere Observing Facilities, and outreach to earth and oceans communities.