CIF21 DIBBs: EI: Vizier, Streamlined Data Curation

大数据有望对我们生活的许多方面产生积极影响，但收集数据来回答问题或推导预测模型可能是具有挑战性的。数据科学家通常必须经历多轮筛选，即对数据进行组织、提炼、清理和合并，然后才能进行分析。精选通常是缓慢和昂贵的，但对于获得有用和可信的答案来说是必不可少的。该项目开发了一个名为Vizier的软件工具，旨在简化数据管理，并使没有计算机科学专业知识的领域专家能够管理自己的数据。更轻松的管理通过使范围广泛的用户能够提高数据质量来放大大数据的价值，从而使政府、行业和科学界的各种数据驱动型工作受益。Vizier具有结合笔记本和电子表格元素的直观界面，使分析师能够快速查看、编辑和修订数据。这一功能得到了自动化数据清理步骤的框架的补充，该框架与手动管理操作无缝集成。Vizier的核心是一个用于管理管理工作流和数据的不确定性和来源的系统，使用户能够跟踪更高级别的管理操作以及跟踪数据的沿袭。通过透明地维护用户所有操作的历史及其对管理数据的影响，Vizier实现了无遗憾的探索和管理，其中对数据及其传递影响的任何更改都可以撤消。通过学习过去的管理历史，该系统还将能够根据上下文为用户提供额外管理行动的建议。这项由高级网络基础设施司颁发的奖项由NSF社会、行为和经济科学局(社会科学和经济科学司)联合支持。

Big Data promises to have a positive impact on many aspects of our lives, but assembling the data to answer questions or derive predictive models can be challenging. Data scientists must typically go through multiple rounds of curation, or 'wrangling,' where data are organized, refined, cleaned up, and merged together before they can be analyzed. Curation is often slow and costly, but is essential for obtaining useful and trustworthy answers. This project develops a software tool called Vizier that aims to streamline data curation and enable domain experts who do not have computer science expertise to curate their own data. Easier curation magnifies the value of big data by enabling a wide range of users to improve data quality, and in doing so benefits numerous types of data-driven work in government, industry, and science.Vizier features an intuitive interface combining elements of notebooks and spreadsheets, allowing analysts to quickly see, edit, and revise data. This capability is complemented by a framework for automated data cleaning steps that are seamlessly integrated with manual curation operations. The heart of Vizier is a system for managing uncertainty and provenance of curation workflows and data, enabling the user to keep track of higher-level curation operations as well as track the lineage of data. By transparently maintaining the history of all the user's actions and their effect on the curated data, Vizier enables regret-free exploration and curation where any changes to the data and their transitive effects can be undone. By learning from past curation histories, the system will also be able to provide users with context-dependent recommendations for additional curation actions.This award by the Advanced Cyberinfrastructure Division is jointly supported by the NSF Directorate for Social, Behavioral and Economic Sciences (Division of Social and Economic Sciences).