Adaptive Bayesian Models for Entity Resolution with Heterogeneous Data

整合来自多个来源的信息的方法变得至关重要，以便对数据流进行完整和准确的数据分析。从噪声数据中合并和删除重复信息的过程称为实体解析或记录链接。实体解析任务普遍存在于许多领域，包括公共卫生、人权、官方统计、社会网络、欺诈检测和国家安全等。尽管近年来实体解析的概率方法变得更加普遍，但对于大数据集来说，同样在计算上容易处理和可扩展的原则性方法是有限的。该项目旨在开发适用于具有不同类型数据的实体解析任务的贝叶斯模型和高效计算算法。这些方法将通过开源软件向从业者和其他研究人员开放。具有多个文件的实体解析可以被视为其中代表相同潜在实体的相似记录被分组在一起的集群任务。在这方面，预计会有大量的小集群或微集群。将探索以下三种一般研究途径：(A)随机分区的自适应先验分布，这些先验分布显示微聚类特性，并允许在不同的尺度上直接合并先验信息；(B)适用于实体解析任务的综合贝叶斯模型，具有根据可用信息的性质很容易调整的社会网络数据；(C)大数据上的实体解析应用的模型加速的计算算法。将探索各种马尔可夫链蒙特卡罗算法和在实体解析的微聚类设置中用于后验推断的高效替代方案，以克服高维离散空间中贝叶斯推理的已知实用限制。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Methods for integrating information from multiple sources have become critical in order to perform complete and accurate data analyses on streams of data. The process of merging and removing duplicate information from noisy data is known as entity resolution or record linkage. Entity resolution tasks are prevalent in many areas, including public health, human rights, official statistics, social networks, fraud detection, and national security, among others. Although probabilistic approaches for entity resolution have become more pervasive in recent years, principled approaches that are also computationally tractable and scalable for large data sets are limited. This project aims to develop Bayesian models and efficient computational algorithms suited for entity resolution tasks with heterogeneous types of data. The methods will be made accessible to practitioners and other researchers through open-source software. Entity resolution with multiple files can be treated as a clustering task in which similar records that represent the same latent entity are grouped together. In this context, a large number of small clusters or microclusters is expected. The following three general avenues of research will be explored: (a) adaptive prior distributions for random partitions that display microclustering properties and permit straightforward incorporation of prior information at different scales; (b) integrated Bayesian models suited for entity resolution tasks with social network data that are easily adaptable according to the nature of the available information; and (c) computational algorithms for model acceleration of entity resolution applications on big data. A variety of Markov Chain Monte Carlo algorithms and efficient alternatives for posterior inference in the microclustering setting of entity resolution will be explored to overcome the known practical limitations of Bayesian inference in high-dimensional discrete spaces.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.