III : Medium: Collaborative Research: From Open Data to Open Data Curation

受重视机构开放性和透明度的社会趋势的推动，开放数据正在以超过我们处理能力的速度产生和共享。许多政府和私人机构正在采用开放数据原则，该原则规定共享数据是完整的，准确的，及时的。 这些属性使这些数据对数据科学家、记者和公众具有巨大的价值。 当开放数据得到有效利用时，数据科学家可以探索和分析开放资源，这反过来又使他们能够调查公共政策，创造新的科学知识，并发现对社会，科学或经济举措有用的新（隐藏）价值。 尽管开放数据运动成功地实现了让数据可访问的雄心，但它并没有成功地让这些有价值的数据易于使用。该项目的首要目标是解决这一缺点。在该项目中，我们提出了开放数据策展的愿景-开放、透明且可解释的数据策展。 Open Data Curation使用一种按需集成范式，涵盖数据发现、数据清理和链接以及数据集成。 我们的愿景是使用户能够查询存储在数据存储库中的异构数据，只需最少的前期工作。 用户可以在查询中引用数据中不存在的概念和属性。 按需集成系统（ODIS）通过自动确定什么数据可以被转换和集成以提供用于所请求的概念的数据来响应这样的请求。 在社会影响方面，该项目将提供算法创新，使开放数据湖上有效、直观的按需集成成为现实。 我们的解决方案将使用真实的开放数据，并将对开放数据的有时古怪且总是多样的特征具有鲁棒性。 我们认为，人们对数据集成和策展的看法需要发生深刻的转变，以推动数据科学革命，这场革命正受到不连贯的数据策展的阻碍-这项任务仍然被认为是最耗时，最烦人，和错误-该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准。

Motivated by societal trends that value institutional openness and transparency, open data is being produced and shared at a speed that surpasses our ability to process it. Many governmental and private institutions are adopting Open Data Principles that state that the shared data is complete, accurate, and timely. These properties make this data of great value to data scientists, journalists, and the public. When Open Data is used effectively, data scientists can explore and analyze open resources, which in turn allows them to investigate public policy, create new scientific knowledge, and discover new (hidden) value useful for social, scientific, or economic initiatives. Though the open data movement has succeeded in its ambition of making data accessible, it has not succeed in making this valuable data easy to use. The overarching goal of this project is to address this shortcoming.In this project, we present a vision for Open Data Curation - data curation that is open, transparent, and explainable. Open Data Curation uses an on-demand integration paradigm that spans data discovery, data cleaning and linking, and data integration. Our vision is to enable users to query heterogeneous data stored in a data repository with minimal up-front effort. Users can reference concepts and attributes in their queries that do not exist in the data. An on-demand integration system (ODIS) responds to such requests by automatically determining what data could be transformed and integrated to provide data for a requested concept. In terms of societal impact, the project will provide the algorithmic innovations to make effective, intuitive on-demand integration over open data lakes a reality. Our solutions will use real open data and will be robust to the sometimes quirky, and always diverse, characteristics of open data. We believe a profound shift in how people think about data integration and curation is needed to fuel the data science revolution which is being held back by incoherent data curation - a task that is still considered one of the most time consuming, annoying, and error-prone in data science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.