In Search of an Entity Resolution OASIS: Optimal Asymptotic Sequential Importance Sampling
In Search of an Entity Resolution OASIS: Optimal Asymptotic Sequential Importance Sampling
复制标题
寻找实体解析 OASIS：最优渐进顺序重要性采样
DOI：
--
复制
发表时间：
2017
影响因子：
2.5
通讯作者：
Benjamin I. P. Rubinstein
中科院分区：
文献类型：
--
作者：
Neil G. Marchant;Benjamin I. P. Rubinstein
实体解析 (ER) 给评估方法带来了独特的挑战。虽然众包平台获得了事实真相，但合理的采样方法必须推动标签工作。在 ER 中，在寻求统计上一致的估计值以进行严格评估时，匹配和不匹配记录之间的极端类别不平衡可能会导致巨大的标签要求。本文通过 OASIS 算法解决了这一重要挑战：用于 ER 评估的采样器和 F 测量估计器。 OASIS 从（有偏差的）工具分布中抽取样本，选择这些样本是为了确保估计量具有最佳渐近方差。随着新标签的收集，OASIS 通过注释器预言机的贝叶斯潜变量模型更新此工具分布，以快速关注未标记的项目，从而提供更多信息。我们证明了 F 度量、精度、召回率的估计结果收敛于真实总体值。对各种 ER 数据集上的采样方法进行的彻底比较表明，标签显着减少了高达 83%，且估计准确性没有损失。
Entity resolution (ER) presents unique challenges for evaluation methodology. While crowdsourcing platforms acquire ground truth, sound approaches to sampling must drive labelling efforts. In ER, extreme class imbalance between matching and non-matching records can lead to enormous labelling requirements when seeking statistically consistent estimates for rigorous evaluation. This paper addresses this important challenge with the OASIS algorithm: a sampler and F-measure estimator for ER evaluation. OASIS draws samples from a (biased) instrumental distribution, chosen to ensure estimators with optimal asymptotic variance. As new labels are collected OASIS updates this instrumental distribution via a Bayesian latent variable model of the annotator oracle, to quickly focus on unlabelled items providing more information. We prove that resulting estimates of F-measure, precision, recall converge to the true population values. Thorough comparisons of sampling methods on a variety of ER datasets demonstrate significant labelling reductions of up to 83% without loss to estimate accuracy.