Approximating and Reasoning about Data Provenance

关于数据来源的近似和推理

• 批准号: 9243763
• 负责人: Zachary Ives
• 金额: $ 15.36万
• 依托单位: UNIVERSITY OF PENNSYLVANIA
• 依托单位国家: 美国
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-15 至 2018-05-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9243763
• 关键词: Adoption; Affect; Area; Big Data; Code; Communities; Complex; Costs and Benefits; Data; Data Analyses; Data Provenance; Data Quality; Ensure; Evaluation; Goals; Graph; Health; Human; Institution; Laboratories; Language; Lead; Medicine; Modeling; Outcome; Perception; Procedures; Process; Provider; Pythons; Reproducibility; Research; Research Infrastructure; Running; Sampling; Science; Site; Source; Specific qualifier value; Staging; Standardization; System; Techniques; Technology; Testing; Time; Uncertainty; Weight; base; computerized data processing; functional genomics; insight; instrument; next generation sequencing; operation; reconstruction; research study; success; tool

 DESCRIPTION (provided by applicant): In many Big Data applications today, such as Next-Generation Sequencing, data processing pipelines are highly complex, span multiple institutions, and include many human and computational steps. The pipelines evolve over time and vary across institutions, so it is difficult to track and reason about the processing pipelines to ensure consistency and correctness of results. Provenance-enabled scientific workflow systems promise to aid here - yet such workflow systems are often avoided due to perceptions of inflexibility, lack of good provenance analytics tools, and emphasis on supporting the data consumer rather than producer. We propose to better incentivize the adoption of workflow and other provenance tracking tools: (1) Instead of requiring a single workflow system across the entire pipeline, which can be inflexible, we allow for integration across multiple autonomous systems (provenance- enabled workflow systems, provenance tracking systems for languages like Python and R, etc.), and even across steps performed without any provenance tracking at all. (2) We develop provenance reasoning capabilities specifically useful to the data provider, such as provenance analytics across time, sites, and users; finding the code modules that best explain why two results are different; regression testing to determine whether a code change would affect prior results; and reconstructing missing provenance for steps that were not captured. These capabilities are expected to lead to wider tracking of data provenance, and ultimately to more consistent, reproducible, and reliable science. We will validate this hypothesis through the evaluation of our technologies within a Next-Generation Sequencing pipeline run by one of the PIs with collaborators at other institutions.

 描述（由申请人提供）：在当今的许多大数据应用中，例如下一代测序，数据处理管道非常复杂，跨越多个机构，并且包括许多人工和计算步骤。管道随着时间的推移而发展，并且在不同的机构中有所不同，因此很难跟踪和推理处理管道 确保结果的一致性和正确性。支持出处的科学工作流系统有望在这里提供帮助-但由于对可扩展性的看法，缺乏良好的出处分析工具，以及强调支持数据消费者而不是生产者，这种工作流系统通常被避免。我们建议更好地激励工作流和其他出处跟踪工具的采用：（1）我们允许跨多个自治系统（支持出处的工作流系统，Python和R等语言的出处跟踪系统等）进行集成，而不是要求整个管道中的单个工作流系统，这可能是不灵活的，甚至在没有任何出处跟踪的情况下执行的步骤之间。 (2)我们开发了对数据提供者特别有用的出处推理功能，例如跨时间、站点和用户的出处分析;找到最能解释为什么两个结果不同的代码模块;回归测试以确定代码更改是否会影响先前的结果;以及为未捕获的步骤重建缺失的出处。这些功能有望导致更广泛的数据来源跟踪，并最终实现更一致，可重复和可靠的科学。我们将验证这一假设 通过在下一代测序管道中对我们的技术进行评估，该管道由一名PI与其他机构的合作者运行。