Optimal two-phase sampling design for comparing accuracies of two binary classification rules.
Optimal two-phase sampling design for comparing accuracies of two binary classification rules.
复制标题
用于比较两个二元分类规则的准确性的最佳两阶段采样设计。
DOI：
10.1002/sim.5946
复制
发表时间：
2014
影响因子：
2
通讯作者：
Grannis,Shaun
中科院分区：
文献类型：
--
作者：
Xu,Huiping;Hui,SiuL;Grannis,Shaun
在本文中，我们考虑了比较两种二元分类规则的性能的设计，例如，两种记录链接算法或两种筛选测试。当样本中的每个单位都有金标准时，或者在两个阶段的研究中，当金标准仅在第二阶段的子样本中使用固定抽样方案确定时，统计方法可以很好地用于比较这些准确性测量。然而，这些方法并不试图优化抽样方案以最小化感兴趣的估计量的方差。在比较两种分类规则的性能时，主要关注的参数是灵敏度、特异性和阳性预测值的差异。我们导出了这些参数估计的分析方差公式，并用它们来获得最优抽样设计。通过比较最优抽样与简单随机抽样和比例分配的实证研究，评价了最优抽样设计的效率。实证研究结果表明，最优抽样设计在估计敏感性和特异性差异方面是相似的，并且在结果不一致的受试者的过样本和结果一致的受试者的欠样本中都实现了大量的方差减少。当没有关于个体敏感性和特异性的先验知识，或研究人群中真阳性发现的普遍性时，建议使用启发式规则。将最优采样应用于记录链接中的一个实际示例，以评估两种匹配算法在分类精度上的差异。版权所有©2013 John Wiley & Sons， Ltd
In this paper, we consider the design for comparing the performance of two binary classification rules, for example, two record linkage algorithms or two screening tests. Statistical methods are well developed for comparing these accuracy measures when the gold standard is available for every unit in the sample, or in a two‐phase study when the gold standard is ascertained only in the second phase in a subsample using a fixed sampling scheme. However, these methods do not attempt to optimize the sampling scheme to minimize the variance of the estimators of interest. In comparing the performance of two classification rules, the parameters of primary interest are the difference in sensitivities, specificities, and positive predictive values. We derived the analytic variance formulas for these parameter estimates and used them to obtain the optimal sampling design. The efficiency of the optimal sampling design is evaluated through an empirical investigation that compares the optimal sampling with simple random sampling and with proportional allocation. Results of the empirical study show that the optimal sampling design is similar for estimating the difference in sensitivities and in specificities, and both achieve a substantial amount of variance reduction with an over‐sample of subjects with discordant results and under‐sample of subjects with concordant results. A heuristic rule is recommended when there is no prior knowledge of individual sensitivities and specificities, or the prevalence of the true positive findings in the study population. The optimal sampling is applied to a real‐world example in record linkage to evaluate the difference in classification accuracy of two matching algorithms. Copyright © 2013 John Wiley & Sons, Ltd.