SI2-SSI: Collaborative Research: Bringing End-to-End Provenance to Scientists

可重复性是科学进步的基石。从历史上看，科学家通过在实验中使用的实验方法的公式化描述来使他们的工作可重复。在一个计算科学的时代，这样的描述不再足以描述科学的方法论。相反，科学的可重复性依赖于对用于进行研究的数据和程序的精确和可操作的描述。出处是描述数字工件如何进入其当前状态的名称。出处包括对实验输入数据和应用于该数据的程序或程序的精确说明。大多数计算平台不记录这些数据的来源，因此很难确保再现性。该项目通过开发工具来解决这个问题，这些工具可以透明地自动捕获数据来源，作为科学家正常计算工作流程的一部分。一个由计算机科学家和生态学家组成的跨学科团队已经走到一起，开发工具来促进数据来源的捕获、管理和查询——数据来源是数字工件如何进入其当前状态的历史。这样的数据来源提高了科学结果的透明度、可靠性和可重复性。大多数现有的来源系统要求用户学习专门的工具和术语，并且无法整合来自不同来源的来源；这些都是领域科学家采用的严重障碍。该项目包括端到端系统（eeProv）的设计、开发、部署和评估，该系统涵盖了从领域科学家的原始数据分析到使用公共工具在公共框架中对结果来源进行管理和分析的活动范围。这个项目利用并集成了以下方面的开发工作：(1)一个用于从科学家实际使用的计算环境（R统计语言）生成来源的新兴系统；(2)一个利用语言库和数据库适配器来存储和管理几乎任何来源的来源的新兴系统。要实现这一建议的目标，需要进行基础研究，解决在不同环境中收集的来源之间的语义差距，在编程语言级别捕获详细的来源，精确定义不同用例所需的来源方面，并使科学家能够访问来源。

Reproducability is the cornerstone of scientific progress. Historically, scientists make their work reproducible by including a formulaic description of the experimental methodology used in an experiment. In an age of computational science, such descriptions no longer adequately describe scientific methodology. Instead, scientific reproducibility relies on a precise and actionable description of the data and programs used to conduct the research. Provenance is the name given to the description of how a digital artifact came to be in its present state. Provenance includes a precise specification of an experiment's input data and the programs or procedures applied to that data. Most computational platforms do not record such data provenance, making it difficult to ensure reproducability. This project addresses this problem through the development of tools that transparently and automatically capture data provenance as part of a scientist's normal computational workflow.An interdisciplinary team of computer scientists and ecologists have come together to develop tools to facilitate the capture, management, and query of data provenance -- the history of how a digital artifact came to be in its present state. Such data provenance improves the transparency, reliability, and reproducibility of scientific results. Most existing provenance systems require users to learn specialized tools and jargon and are unable to integrate provenance from different sources; these are serious obstacles to adoption by domain scientists. This project includes the design, development, deployment, and evaluation of an end-to-end system (eeProv) that encompasses the range of activity from original data analysis by domain scientists to management and analysis of the resulting provenance in a common framework with common tools. This project leverages and integrates development efforts on (1) an emerging system for generating provenance from a computing environment that scientists actually use (the R statistical language) with (2) an emerging system that utilizes a library of language and database adapters to store and manage provenance from virtually any source. Accomplishing the goals of this proposal requires fundamental research in resolving the semantic gap between provenance collected in different environments, capturing detailed provenance at the level of a programming language, defining precisely aspects of provenance required for different use cases, and making provenance accessible to scientists.