EAGER: Lifecycle Management of Collaborative Analysis Workflows through Provenance Capture and Analysis

数据驱动的方法和产品已经显示出巨大的前景，并且在各种社区中变得越来越普遍，包括科学，教育，经济，政府以及社会和网络分析。这一趋势通常被称为“大数据”或“数据科学”，导致迫切需要可持续和可扩展的工具，以促进端到端的协作数据分析过程;这个过程通常是特别的，典型地具有高度非结构化的数据集、不同工具和技术的合并、团队成员之间的重要来回，通过反复试验来确定正确的分析工具、算法、模型和参数。 尽管在开发工具以执行特定数据分析任务方面有很多先前和正在进行的工作，但没有简单的方法来捕获和推理特定的数据科学管道，其中许多管道通常分布在一系列分析脚本中。关于数据集如何生成的元数据或出处信息，包括用于生成它们的程序或脚本和/或任何关键参数的值，通常会丢失。同样，很难跟踪数据集之间的任何依赖关系，或者关于它们如何随时间演变的信息。该项目正在构建一个统一的出处和元数据管理系统，以支持大数据应用中出现的复杂协作“数据科学工作流”的端到端生命周期管理。该系统具有灵活和直观的数据模型，可以捕获各种不同类型的数据和元数据，包括版本和出处信息，衍生信息，在实验或建模过程中使用的参数，收集的统计数据，以做出决策，分析脚本，注释或标签等，它提供了新颖的机制，使其易于捕获这些信息，对用户的负担最小。该系统还具有丰富的高级特定领域查询语言，可以对这些数据进行统一查询，以及基于Web浏览器的可视化工具，用于制定查询和探索搜索结果。通过不断分析和利用这些来源信息，该系统还实现了许多新功能，包括：搜索给定任务的相关数据科学工作流或分析脚本，比较多个管道的最终结果以识别关键的相似性和差异，以及在模型开发和/或部署期间快速自动检测问题或异常。该系统将改变数据科学家在执行数据分析时管理来源信息和元数据的方式，并使他们能够更快地从数据中获得可操作和有用的见解或知识。通过降低共享和重用他人所做工作的障碍，该系统将带来以前可能无法实现的新见解。该项目为研究生和本科生提供研究机会，并与PI提供的几门本科和研究生课程保持一致。

Data-driven methods and products have shown tremendous promise and are becoming increasingly common in a variety of communities, including science, education, economics, government, and social and web analytics. This trend, popularly referred to as "big data" or "data science", has resulted in a pressing need for sustainable and scalable tools that facilitate the end-to-end collaborative data analysis process; this process is often ad hoc, typically featuring highly unstructured datasets, an amalgamation of different tools and techniques, significant back-and-forth among the members of a team, and trial-and-error to identify the right analysis tools, algorithms, models, and parameters. Although there is much prior and ongoing work on developing tools to perform specific data analysis tasks, there is no easy way to capture and reason about ad hoc data science pipelines, many of which are often spread across a collection of analysis scripts. Metadata or provenance information about how datasets were generated, including the programs or scripts used for generating them and/or values of any crucial parameters, is often lost. Similarly, it is hard to keep track of any dependencies between the datasets, or information about how they evolved over time. This project is building a unified provenance and metadata management system to support end-to-end lifecycle management of complex collaborative "data science workflows" that arise in big data applications. The system features a flexible and intuitive data model that can capture a variety of different types of data and metadata, including versioning and provenance information, derivation information, parameters used during experiments or modeling, statistics gathered to make decisions, analysis scripts, notes or tags, etc. It provides novel mechanisms for making it easy to capture such information with minimal burden on the users. The system also features a rich, high-level domain-specific query language that enables unified querying over such data, as well as a web browser-based visualization tool for formulating queries, and for exploring the search results. By continuously analyzing and exploiting such provenance information, the system also enables a host of new features including: searching for relevant data science workflows or analysis scripts for a given task, comparing end results of multiple pipelines to identify key similarities and differences, and quickly and automatically detecting problems or anomalies during model development and/or deployment. The system will transform the way in which data scientists manage provenance information and metadata while performing data analysis, and will allow them to more quickly derive actionable and useful insights or knowledge from the data. By lowering the barrier to sharing and reusing the work done by others, the system will lead to new insights that may not have been achievable beforehand. This project provides research opportunities for graduate and undergraduate students, and is aligned with several undergraduate and graduate courses offered by the PI.