CAREER: Declarative Uncertainty

数据是混乱的。幸运的是，通过最少的人为干预，良好的数据清理启发式方法可以从大而混乱的数据中产生大多数可靠的、通常可操作的信息。例如，分析人员可以通过使用分类器来预测缺失的属性值，或者通过使用实体解析器来查找和合并重复的记录，从而自动化他们的管理工作流。不幸的是，启发式也很危险，因为启发式策展的结果常常被当作事实。严重的错误，比如人们因为别人的不良信用而被拒绝贷款，12岁的孩子被认定为恐怖分子，或者数十亿美元的投资错误，往往是在低信心或不确定的启发式推断被视为真理时发生的。许多有原则的工具，如概率数据库，已经存在，可以自动跟踪不可靠数据中的潜在错误，但这些工具并不容易使用。因此，分析人员更多地求助于简单地记录潜在的错误，并希望任何使用这些数据的人都能意识到其中的含义。该建议将使数据管理系统能够查询和组织不确定的数据，而不会难以使用。该提案的具体目标是将不确定数据的提问过程与诸如数据为什么不确定，用户希望如何查看查询结果中的不确定性，或者应该使用哪种算法等机械问题解耦。为了启用这种“声明性不确定性管理”，项目团队将构建一个名为Mimir的系统，该系统通过增加数据管理工作流（例如，ETL管道）来虚拟化不确定性，并使用一种来源捕获形式，通过这种形式，启发式可以注册可选输出（例如，模式匹配器可以注册多个潜在匹配）。然后，这个来源可以用来综合数据和查询结果中不确定性的各种不同的物理和视觉表示。为了实现声明性不确定性管理，本提案将解决以下两大类具体问题：(1)选择并有效地构建查询结果中不确定性的定性摘要，以及(2)增强数据库查询编译器和优化器，以支持对不确定数据进行实际、高效的查询处理。欲了解更多信息，请参阅该项目的网页：http://mimirdb.infoThis该奖项反映了美国国家科学基金会的法定使命，并通过基金会的智力价值和更广泛的影响审查标准进行评估，认为值得支持。

Data is messy. Fortunately, with minimal human intervention, good data cleaning heuristics produce mostly reliable, usually actionable information from big, messy data. For instance, analysts might automate their curation workflows by using classifiers to predict missing attribute values, or by using an entity-resolver to find and merge duplicate records. Unfortunately, heuristics are also dangerous, as the result of heuristic curation is often taken as fact. Serious mistakes like people being denied a loan due to someone else's bad credit, 12-year olds being identified as terrorists, or billion dollar investment errors, often result when low-confidence, or uncertain heuristic inferences are treated as truth. Many principled tools like probabilistic databases already exist for automatically tracking potential errors in unreliable data, but these tools are not easy to use. As a result, analysts more often resort to simply documenting potential errors and hoping that anyone using the data will realize the implications. This proposal will enable data management systems that can query and organize uncertain data, without being hard to use. The specific aim of this proposal is to decouple the process of asking questions about uncertain data from mechanical concerns like why the data is uncertain, how the user wants to view uncertainty in query results, or which algorithms should be used. To enable this sort of "declarative uncertainty management," the project team will build on a system called Mimir that virtualizes uncertainty by augmenting data curation workflows (e.g., ETL pipelines) with a form of provenance capture through which heuristics can register alternative outputs (e.g., a schema matcher may register multiple potential matches). This provenance can then be used to synthesize a wide range of different physical and visual representations of uncertainty in data and in query results. To enable declarative uncertainty management, this proposal will address specific problems that fall into two general categories: (1) selecting and efficiently constructing qualitative summaries of uncertainty in query results, and (2) enhancing database query compilers and optimizers to support practical, efficient query processing over uncertain data. For further information see the project web page: http://mimirdb.infoThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.