Adaptive Bayesian Models for Entity Resolution with Heterogeneous Data

为了对数据流进行完整和准确的数据分析，整合来自多个来源的信息的方法变得至关重要。从噪声数据中合并和删除重复信息的过程称为实体解析或记录链接。实体解析任务在许多领域都很普遍，包括公共卫生、人权、官方统计、社交网络、欺诈检测和国家安全等。虽然近年来实体解析的概率方法变得越来越普遍，但对于大型数据集来说，计算上易于处理和可扩展的原则性方法是有限的。该项目旨在开发贝叶斯模型和高效的计算算法，适用于具有异构类型数据的实体解析任务。这些方法将通过开放源码软件提供给从业人员和其他研究人员。 具有多个文件的实体解析可以被视为聚类任务，其中表示相同潜在实体的类似记录被分组在一起。在这种情况下，预计会有大量的小簇或微簇。将探讨以下三种一般研究途径：（a）随机分区的自适应先验分布，显示微聚类特性，并允许在不同尺度上直接合并先验信息;（B）适用于实体解析任务的集成贝叶斯模型与社会网络数据，易于根据可用信息的性质进行调整;以及（c）用于大数据上的实体解析应用的模型加速的计算算法。各种马尔可夫链蒙特卡罗算法和有效的替代品后的推理在实体分辨率的微聚类设置将探讨克服已知的实际限制贝叶斯推理在高维discrete spaces.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Methods for integrating information from multiple sources have become critical in order to perform complete and accurate data analyses on streams of data. The process of merging and removing duplicate information from noisy data is known as entity resolution or record linkage. Entity resolution tasks are prevalent in many areas, including public health, human rights, official statistics, social networks, fraud detection, and national security, among others. Although probabilistic approaches for entity resolution have become more pervasive in recent years, principled approaches that are also computationally tractable and scalable for large data sets are limited. This project aims to develop Bayesian models and efficient computational algorithms suited for entity resolution tasks with heterogeneous types of data. The methods will be made accessible to practitioners and other researchers through open-source software. Entity resolution with multiple files can be treated as a clustering task in which similar records that represent the same latent entity are grouped together. In this context, a large number of small clusters or microclusters is expected. The following three general avenues of research will be explored: (a) adaptive prior distributions for random partitions that display microclustering properties and permit straightforward incorporation of prior information at different scales; (b) integrated Bayesian models suited for entity resolution tasks with social network data that are easily adaptable according to the nature of the available information; and (c) computational algorithms for model acceleration of entity resolution applications on big data. A variety of Markov Chain Monte Carlo algorithms and efficient alternatives for posterior inference in the microclustering setting of entity resolution will be explored to overcome the known practical limitations of Bayesian inference in high-dimensional discrete spaces.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.