III : Medium: Collaborative Research: From Open Data to Open Data Curation

III：媒介：协作研究：从开放数据到开放数据管理

• 批准号: 2107107
• 负责人: Boris Glavic
• 金额: $ 37.5万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107107&HistoricalAwards=false
• 关键词: III; Medium; Collaborative; Research; Open

Motivated by societal trends that value institutional openness and transparency, open data is being produced and shared at a speed that surpasses our ability to process it. Many governmental and private institutions are adopting Open Data Principles that state that the shared data is complete, accurate, and timely. These properties make this data of great value to data scientists, journalists, and the public. When Open Data is used effectively, data scientists can explore and analyze open resources, which in turn allows them to investigate public policy, create new scientific knowledge, and discover new (hidden) value useful for social, scientific, or economic initiatives. Though the open data movement has succeeded in its ambition of making data accessible, it has not succeed in making this valuable data easy to use. The overarching goal of this project is to address this shortcoming.In this project, we present a vision for Open Data Curation - data curation that is open, transparent, and explainable. Open Data Curation uses an on-demand integration paradigm that spans data discovery, data cleaning and linking, and data integration. Our vision is to enable users to query heterogeneous data stored in a data repository with minimal up-front effort. Users can reference concepts and attributes in their queries that do not exist in the data. An on-demand integration system (ODIS) responds to such requests by automatically determining what data could be transformed and integrated to provide data for a requested concept. In terms of societal impact, the project will provide the algorithmic innovations to make effective, intuitive on-demand integration over open data lakes a reality. Our solutions will use real open data and will be robust to the sometimes quirky, and always diverse, characteristics of open data. We believe a profound shift in how people think about data integration and curation is needed to fuel the data science revolution which is being held back by incoherent data curation - a task that is still considered one of the most time consuming, annoying, and error-prone in data science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在重视机构开放性和透明度的社会趋势的推动下，开放数据正在以超出我们处理能力的速度产生和共享。许多政府和私人机构正在采用开放数据原则，声明共享的数据是完整、准确和及时的。这些属性使这些数据对数据科学家、记者和公众具有很大的价值。当开放数据被有效地使用时，数据科学家可以探索和分析开放资源，这反过来又允许他们调查公共政策，创造新的科学知识，并发现对社会，科学或经济举措有用的新（隐藏）价值。虽然开放数据运动已经成功地实现了数据可访问性的目标，但它还没有成功地使这些有价值的数据易于使用。这个项目的首要目标就是解决这个缺点。在这个项目中，我们提出了一个开放数据管理的愿景——开放、透明和可解释的数据管理。开放数据管理使用一种按需集成范例，涵盖数据发现、数据清理和链接以及数据集成。我们的愿景是使用户能够以最少的前期工作查询存储在数据存储库中的异构数据。用户可以在查询中引用数据中不存在的概念和属性。按需集成系统（ODIS）通过自动确定可以转换和集成哪些数据来为所请求的概念提供数据，从而响应此类请求。在社会影响方面，该项目将提供算法创新，使开放数据湖上有效、直观的按需集成成为现实。我们的解决方案将使用真正的开放数据，并将对开放数据有时古怪但总是多样化的特征保持稳健。我们认为，人们对数据整合和管理的看法需要发生深刻的转变，以推动数据科学革命，这一革命被不连贯的数据管理所阻碍——数据管理仍然被认为是数据科学中最耗时、最烦人、最容易出错的任务之一。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

CaJaDE: explaining query results by augmenting provenance with context

CaJaDE：通过使用上下文增强来源来解释查询结果

• DOI: 10.14778/3554821.3554852
• 发表时间: 2022
• 期刊: Proceedings of the VLDB Endowment
• 影响因子: 2.5
• 作者: Li, Chenjie;Lee, Juseung;Miao, Zhengjie;Glavic, Boris;Roy, Sudeepa
• 通讯作者: Roy, Sudeepa

Interpretable Data-Based Explanations for Fairness Debugging

• DOI: 10.1145/3514221.3517886
• 发表时间: 2021-12
• 期刊: Proceedings of the 2022 International Conference on Management of Data
• 作者: Romila Pradhan;Jiongli Zhu;Boris Glavic;Babak Salimi

Trends in Explanations: Understanding and Debugging Data-driven Systems

• DOI: 10.1561/1900000074
• 发表时间: 2021
• 期刊: Found. Trends Databases
• 作者: Boris Glavic;A. Meliou;Sudeepa Roy

Debugging missing answers for spark queries over nested data with breadcrumb

使用面包屑调试嵌套数据上 Spark 查询的缺失答案

• DOI: 10.14778/3476311.3476331
• 发表时间: 2021
• 期刊: Proceedings of the VLDB Endowment
• 影响因子: 2.5
• 作者: Diestelkämper, Ralf;Lee, Seokki;Glavic, Boris;Herschel, Melanie
• 通讯作者: Herschel, Melanie

Hybrid Query and Instance Explanations and Repairs

混合查询和实例解释和修复

• DOI: 10.1145/3543873.3587565
• 发表时间: 2023
• 期刊: TaPP workshop - WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023
• 作者: Lee, Seokki;Glavic, Boris;Chapman, Adriane;Ludäscher, Bertram
• 通讯作者: Ludäscher, Bertram