SBIR Phase I: Statistical Inference for Advanced Entity Resolution

这个小型企业创新研究（SBIR）第一阶段项目解决了整合来自众多数据源的有关命名实体（如人员、公司和产品）的信息的问题。 集成来自多个源的关于实体的信息可能是困难的，因为源可能使用不同的格式和术语来描述相同的实体，这一问题被称为“实体解析”。 大多数现有的商业企业系统依赖于基于规则的匹配技术来进行实体解析。 该项目研究统计学习技术，使系统能够估计匹配的概率，而不是基于自组织规则或权重计算分数。由于该方法基于合理的统计原理，并使用从大型数据集汇编的证据，因此它可以产生比现有方法更准确的结果。此外，当处理具有高度可变、缺失或噪声属性的数据（例如从Web站点提取的数据）时，这些优势会被放大。该项目更广泛的影响/商业潜力在于使企业能够执行更准确和可靠的数据集成。 许多潜在的目标市场需要更好的技术来集成有关企业、产品、人员、位置和其他实体的信息。 这种能力对美国一些最大的公司和机构至关重要，从搜索引擎到美国情报和执法机构，再到金融机构。 特别是，大型企业通常难以利用从新闻、外语数据源和社交媒体提取的数据，因为提取的数据是嘈杂的并且结构不好。该项目开发的技术将帮助企业利用Web上不断增长的信息量，以便他们能够利用将人、公司和其他实体联系起来的关系网络，更好地为客户服务。

This Small Business Innovation Research (SBIR) Phase I project addresses the problem of integrating information about named entities, such as people, companies, and products, from numerous data sources. Integrating information about entities from multiple sources can be difficult because sources may use different formats and terminology to describe the same entity, a problem referred to as "entity resolution". Most existing commercial enterprise systems rely on rule-based matching techniques for entity resolution. This project investigates statistical learning techniques that allow a system to estimate the probability of a match, rather than computing a score based on ad-hoc rules or weights. Because the approach is based on sound statistical principles and uses evidence compiled from large datasets, it can produce more accurate results than existing methods. Moreover, these advantages are amplified when handling data that that has highly variable, missing or noisy attributes, such as data extracted from Web sites. The broader impact/commercial potential of this project lies in enabling enterprises to perform more accurate and reliable data integration. The are many potential target markets that need better technology for integrating information about businesses, products, people, locations, and other entities. This capability is critical for some of the nation's largest companies and institutions, from search engines, to the U.S. Intelligence and law enforcement community, to financial institutions. In particular, large enterprises often have difficulty utilizing data extracted from news, foreign language data sources, and social media, because the extracted data is noisy and not-well structured. The technology developed in this project will help enterprises make use of the growing amount of information on the Web, so that they can take advantage of the network of relationships that link people, companies, and other entities to serve their customers better.