CAREER: Declarative Uncertainty

数据很混乱。 幸运的是，在最少的人为干预下，良好的数据清理算法可以从大而杂乱的数据中产生大多数可靠的，通常是可操作的信息。 例如，分析师可以通过使用分类器来预测缺失的属性值，或者通过使用实体解析器来查找和合并重复的记录，来自动化他们的策展工作流。 不幸的是，启发式管理也是危险的，因为启发式管理的结果往往被当作事实。 严重的错误，如人们被拒绝贷款，由于别人的不良信用，12岁的奥尔兹被确定为恐怖分子，或数十亿美元的投资错误，往往导致低信心，或不确定的启发式推理被视为真理。 许多有原则的工具，如概率数据库，已经存在，用于自动跟踪不可靠数据中的潜在错误，但这些工具并不容易使用。 因此，分析师更经常地诉诸于简单地记录潜在的错误，并希望任何使用数据的人都能意识到其中的含义。 该提案将使数据管理系统能够查询和组织不确定的数据，而不会难以使用。该提案的具体目标是将询问有关不确定数据的问题的过程与机械问题（例如数据为什么不确定，用户希望如何查看查询结果中的不确定性，或者应该使用哪些算法）分离。 为了实现这种“声明式不确定性管理”，项目团队将构建一个名为Mimir的系统，该系统通过增强数据管理工作流（例如，ETL流水线），通过它，代理可以注册替代输出（例如，模式匹配器可以注册多个潜在的匹配）。 然后，这种起源可以用于合成数据和查询结果中的不确定性的各种不同的物理和视觉表示。 为了实现声明式不确定性管理，该建议将解决落入两个一般类别的特定问题：（1）选择和有效地构建查询结果中的不确定性的定性摘要，以及（2）增强数据库查询编译器和优化器，以支持对不确定数据的实用，高效的查询处理。欲了解更多信息，请参阅项目网页：http://mimirdb.infoThis奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Data is messy. Fortunately, with minimal human intervention, good data cleaning heuristics produce mostly reliable, usually actionable information from big, messy data. For instance, analysts might automate their curation workflows by using classifiers to predict missing attribute values, or by using an entity-resolver to find and merge duplicate records. Unfortunately, heuristics are also dangerous, as the result of heuristic curation is often taken as fact. Serious mistakes like people being denied a loan due to someone else's bad credit, 12-year olds being identified as terrorists, or billion dollar investment errors, often result when low-confidence, or uncertain heuristic inferences are treated as truth. Many principled tools like probabilistic databases already exist for automatically tracking potential errors in unreliable data, but these tools are not easy to use. As a result, analysts more often resort to simply documenting potential errors and hoping that anyone using the data will realize the implications. This proposal will enable data management systems that can query and organize uncertain data, without being hard to use. The specific aim of this proposal is to decouple the process of asking questions about uncertain data from mechanical concerns like why the data is uncertain, how the user wants to view uncertainty in query results, or which algorithms should be used. To enable this sort of "declarative uncertainty management," the project team will build on a system called Mimir that virtualizes uncertainty by augmenting data curation workflows (e.g., ETL pipelines) with a form of provenance capture through which heuristics can register alternative outputs (e.g., a schema matcher may register multiple potential matches). This provenance can then be used to synthesize a wide range of different physical and visual representations of uncertainty in data and in query results. To enable declarative uncertainty management, this proposal will address specific problems that fall into two general categories: (1) selecting and efficiently constructing qualitative summaries of uncertainty in query results, and (2) enhancing database query compilers and optimizers to support practical, efficient query processing over uncertain data. For further information see the project web page: http://mimirdb.infoThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.