EAGER: Lifecycle Management of Collaborative Analysis Workflows through Provenance Capture and Analysis

点击翻译按钮获取中文摘要

Data-driven methods and products have shown tremendous promise and are becoming increasingly common in a variety of communities, including science, education, economics, government, and social and web analytics. This trend, popularly referred to as "big data" or "data science", has resulted in a pressing need for sustainable and scalable tools that facilitate the end-to-end collaborative data analysis process; this process is often ad hoc, typically featuring highly unstructured datasets, an amalgamation of different tools and techniques, significant back-and-forth among the members of a team, and trial-and-error to identify the right analysis tools, algorithms, models, and parameters. Although there is much prior and ongoing work on developing tools to perform specific data analysis tasks, there is no easy way to capture and reason about ad hoc data science pipelines, many of which are often spread across a collection of analysis scripts. Metadata or provenance information about how datasets were generated, including the programs or scripts used for generating them and/or values of any crucial parameters, is often lost. Similarly, it is hard to keep track of any dependencies between the datasets, or information about how they evolved over time. This project is building a unified provenance and metadata management system to support end-to-end lifecycle management of complex collaborative "data science workflows" that arise in big data applications. The system features a flexible and intuitive data model that can capture a variety of different types of data and metadata, including versioning and provenance information, derivation information, parameters used during experiments or modeling, statistics gathered to make decisions, analysis scripts, notes or tags, etc. It provides novel mechanisms for making it easy to capture such information with minimal burden on the users. The system also features a rich, high-level domain-specific query language that enables unified querying over such data, as well as a web browser-based visualization tool for formulating queries, and for exploring the search results. By continuously analyzing and exploiting such provenance information, the system also enables a host of new features including: searching for relevant data science workflows or analysis scripts for a given task, comparing end results of multiple pipelines to identify key similarities and differences, and quickly and automatically detecting problems or anomalies during model development and/or deployment. The system will transform the way in which data scientists manage provenance information and metadata while performing data analysis, and will allow them to more quickly derive actionable and useful insights or knowledge from the data. By lowering the barrier to sharing and reusing the work done by others, the system will lead to new insights that may not have been achievable beforehand. This project provides research opportunities for graduate and undergraduate students, and is aligned with several undergraduate and graduate courses offered by the PI.