SDCI: Data/NMI New/Improvement - Semantic Provenance Capture in Data Ingest Systems (SPCDIS)

数据摄取系统中的语义来源捕获(SPCDIS)在分析仪器数据提供商的需求时，几个挑战显而易见：-数据以更快、更大的数量传入，超出了我们进行充分质量控制的能力。-数据正以新的方式使用，我们经常没有足够的信息来确定它是否适合我们没有预见到的用途。-我们经常无法捕获、表示和传播需要与数据流一起使用的手动生成的信息。-每次我们开发新的仪器时，我们都会开发一个新的数据摄取程序，收集不同的元数据并以不同的方式组织它。因此，它很难用于以前的项目。-事件确定和特征分类的任务繁重，我们直到获得数据后才做这项工作。这些声明指出，缺乏一个全面的、可重复使用的数据获取框架，该框架包括沿数据获取工作流程的语义丰富的注释集，以及用于来源和派生信息的智能存储、传播和检索机制。就本项目而言，出处被定义为：某物的来源或来源、其使用意图、为谁/为什么而产生、其方式、地点感觉、制造、生产或发现的时间、后续所有人的历史，并有足够的详细记录以允许重现。因此，该项目的目标是为数据摄取系统提供可扩展的来源表示。最初，我们将重点放在夏威夷莫纳罗亚太阳天文台由国家大气研究中心高海拔天文台操作的一套日冕物理仪器上。随着时间的推移，我们将瞄准更广泛的太阳和日地物理领域，包括拟议的日冕太阳磁力天文台。这个项目利用了推理网解释框架的创新工作，该框架提供了一套工具来生成、验证、操作、总结和呈现知识来源。此外，我们将使用它的来源、理由和信任表示的中间语言-PML-证明标记语言。要捕获的两个重要概念是数据质量和数据所经过的处理阶段的性质。数据质量的定性和定量编码对于科学家确定感兴趣的数据是否对预期用途有用、适用或足够准确都是非常重要的。来源工作将具有广泛的适用性，因为它将包括针对任何数据摄取系统的与领域无关的部分，以及针对太阳和日地物理的领域识别性模块。该项目预计将产生一个由科学驱动的PML扩展，它将提供科学起源所需的代表性原语。该项目还将通过将元数据添加到相关项目中开发的本体中，对类似的社区和政府项目具有广泛的适用性，从而为社区标准做出贡献。

Semantic Provenance Capture in Data Ingest Systems (SPCDIS)In analyzing the needs of instrument data providers, several challenges areclear:- Data is coming in faster, in greater volumes and outstripping our ability to perform adequate quality control.- Data is being used in new ways and we frequently do not have sufficient information on what happened to the data along the processing stages to determine if it is suitable for a use we did not envision.- We often fail to capture, represent and propagate manually generated information that need to go with the data flows.- Each time we develop a new instrument, we develop a new data ingest procedure and collect different metadata and organize it differently. It is then hard to use with previous projects.- The task of event determination and feature classification is onerous and we don't do it until after we get the data.These statements point to the lack of a comprehensive, re-useable data ingest framework that consists of a semantically rich set of annotations along the data ingest workflow and a smart storage, propagation and retrieval mechanism for the provenance and derivation information. For the purpose of this project, provenance is defined as: the origin or source from which something comes, its intention for use, who/what is was generated for, its manner, sense of place, and time of manufacture, production or discovery, history of subsequent owners, and documented in detail sufficient to allow reproducibility. Thus, the goal of this project is to provide an extensible representation for provenance for data ingest systems. Initially, we limit our focus to the set of solar coronal physics instruments operated at the Mauna Loa Solar Observatory in Hawaii by the High Altitude Observatory, National Center for Atmospheric Research. Over time, we will target the broader area of solar and solar-terrestrial physics, including the proposed Coronal Solar Magnetism Observatory. This project leverages innovative work with the Inference Web explanation framework which provides a set of tools for generating, validating, manipulating, summarizing, and presenting knowledge provenance. In addition we will utilize its Interlingua for provenance, justification, and trust representation - PML - the Proof Markup Language.Two important concepts to be captured are the data quality and nature of the processing stages that the data has passed through. Both qualitative and quantitative encodings of data quality are very important to a scientist determining if the data of interest are useful, applicable or accurate enough for the intended use.The provenance work will have broad applicibility since it will include domain-independent portions geared for any data ingest system as well as a domain-literate module aimed at solar and solar-terrestrial physics. This project is expected to generate a science-driven extension to PML that will provide representational primitives required for scientific provenance. The project will also contribute to community standards by adding meta data to ontologies developed in related projects with a wide degree of applicability to similar community and government programs.