SDCI: Data/NMI New/Improvement - Semantic Provenance Capture in Data Ingest Systems (SPCDIS)

数据摄取系统（SPCDIS）中的语义来源捕获在分析仪器数据提供商的需求时，有几个挑战是显而易见的：—数据传入的速度更快，数量更大，超出了我们执行适当质量控制的能力。-数据正在以新的方式使用，我们经常没有足够的信息来确定数据在处理阶段发生了什么，以确定它是否适合我们没有预见到的用途。-我们经常无法捕获、表示和传播需要随数据流一起运行的手动生成的信息。-每次开发新仪器，我们都会开发新的数据摄取程序，收集不同的元数据，并以不同的方式组织。然后很难与以前的项目一起使用。-事件确定和特征分类的任务很繁重，我们只有在得到数据后才会去做。这些陈述表明，缺乏一个全面的、可重用的数据摄取框架，该框架由数据摄取工作流中语义丰富的一组注释和用于来源和派生信息的智能存储、传播和检索机制组成。为了这个项目的目的，出处被定义为：某物的起源或来源，它的使用意图，它是为谁/什么而产生的，它的方式，地点感，制造，生产或发现的时间，后来的主人的历史，并有足够的详细记录以允许复制。因此，这个项目的目标是为数据摄取系统的来源提供一个可扩展的表示。最初，我们将重点放在夏威夷莫纳罗亚太阳天文台由国家大气研究中心高海拔天文台操作的一套日冕物理仪器上。随着时间的推移，我们将瞄准更广泛的太阳和太阳地球物理领域，包括拟议的日冕太阳磁力观测站。该项目利用了Inference Web解释框架的创新工作，该框架提供了一组用于生成、验证、操作、总结和呈现知识来源的工具。此外，我们将利用它的中间语言进行来源、证明和信任表示——PML——证明标记语言。需要捕获的两个重要概念是数据质量和数据所经过的处理阶段的性质。数据质量的定性和定量编码对于科学家确定感兴趣的数据是否有用、适用或足够准确地用于预期用途非常重要。起源工作将具有广泛的适用性，因为它将包括面向任何数据摄取系统的领域独立部分，以及针对太阳和日地物理的领域读写模块。该项目期望生成一个科学驱动的PML扩展，它将提供科学来源所需的表示原语。该项目还将通过将元数据添加到相关项目开发的本体中，从而为社区标准做出贡献，这些项目广泛适用于类似的社区和政府项目。

Semantic Provenance Capture in Data Ingest Systems (SPCDIS)In analyzing the needs of instrument data providers, several challenges areclear:- Data is coming in faster, in greater volumes and outstripping our ability to perform adequate quality control.- Data is being used in new ways and we frequently do not have sufficient information on what happened to the data along the processing stages to determine if it is suitable for a use we did not envision.- We often fail to capture, represent and propagate manually generated information that need to go with the data flows.- Each time we develop a new instrument, we develop a new data ingest procedure and collect different metadata and organize it differently. It is then hard to use with previous projects.- The task of event determination and feature classification is onerous and we don't do it until after we get the data.These statements point to the lack of a comprehensive, re-useable data ingest framework that consists of a semantically rich set of annotations along the data ingest workflow and a smart storage, propagation and retrieval mechanism for the provenance and derivation information. For the purpose of this project, provenance is defined as: the origin or source from which something comes, its intention for use, who/what is was generated for, its manner, sense of place, and time of manufacture, production or discovery, history of subsequent owners, and documented in detail sufficient to allow reproducibility. Thus, the goal of this project is to provide an extensible representation for provenance for data ingest systems. Initially, we limit our focus to the set of solar coronal physics instruments operated at the Mauna Loa Solar Observatory in Hawaii by the High Altitude Observatory, National Center for Atmospheric Research. Over time, we will target the broader area of solar and solar-terrestrial physics, including the proposed Coronal Solar Magnetism Observatory. This project leverages innovative work with the Inference Web explanation framework which provides a set of tools for generating, validating, manipulating, summarizing, and presenting knowledge provenance. In addition we will utilize its Interlingua for provenance, justification, and trust representation - PML - the Proof Markup Language.Two important concepts to be captured are the data quality and nature of the processing stages that the data has passed through. Both qualitative and quantitative encodings of data quality are very important to a scientist determining if the data of interest are useful, applicable or accurate enough for the intended use.The provenance work will have broad applicibility since it will include domain-independent portions geared for any data ingest system as well as a domain-literate module aimed at solar and solar-terrestrial physics. This project is expected to generate a science-driven extension to PML that will provide representational primitives required for scientific provenance. The project will also contribute to community standards by adding meta data to ontologies developed in related projects with a wide degree of applicability to similar community and government programs.