III: Medium: Quantifying the Unknown Unknowns for Data Integration

随着在线可用数据的数量和种类的爆炸式增长，数据科学家通常的做法是获取和集成不同的数据源，以获得更高质量的结果。但是，即使有一个完全清理和合并的数据集，仍然存在两个基本问题：(1)集成的数据集是完整的吗？(2)任何未知（即未观察到的）数据对查询结果的影响是什么？在这项工作中，该项目将开发和分析技术，以估计未知数据（又称未知未知）对分析查询的影响。这将有助于更好地理解从商业、军事到医疗应用等各个领域存在不完整信息的答案。该项目将开发和利用以下矛盾的统计现象：多次查看某些数据项（跨多个数据集）的能力使人们能够估计从未见过的数据项的参数。因此，该项目将开发新的统计技术，利用重叠的数据集，以及由理论和实验支持的软件。这将使具有重叠的不完整数据集的用户能够主动地“看到未见过的”，并且在许多情况下，就好像他们可以访问任何数据源中未表示的缺失信息一样。该项目还将关注数据验证，以及如何使用多个不可靠的数据源来相互纠正。此外，由于所提出的分析是微妙和新颖的，该项目还将探索如何通过预测的交互式可视化，最好地向用户传达有价值的见解。欲了解更多信息，请参阅该项目的网站：http://unknown-unknowns.cs.brown.edu

As the amount and variety of data available online explodes, it is common practice for data scientists to acquire and integrate disparate data sources to achieve higher quality results. But even with a perfectly cleaned and merged data set, two fundamental questions remain: (1) is the integrated data set complete and (2) what is the impact of any unknown (i.e., unobserved) data on query results? In this work, this project will develop and analyze techniques to estimate the impact of the unknown data (a.k.a., unknown unknowns) for analytical queries. This will help to better understand answers in the presence of incomplete information across fields ranging from business and the military to medical applications.This project will develop and exploit the following paradoxical statistical phenomenon: the ability to see certain data items more than once (across multiple data sets) enables one to estimate parameters of data items that have never been seen at all. This project will therefore develop new statistical techniques which take advantage of overlapping datasets, and software backed by both theory and experiments. This will enable users with overlapping incomplete data sets to actively "see the unseen," and in many cases perform as though they had access to missing information not represented in any of their data sources. The project will also focus on data validation, and how to use multiple unreliable data sources to correct each other. Further, as the proposed analysis is nuanced and novel, the project will also explore how to best convey valuable insights to the user, via interactive visualizations of the predictions. For further information see the project web site at: http://unknown-unknowns.cs.brown.edu