On the bias in the AUC variance estimate.
On the bias in the AUC variance estimate.
复制标题
关于 AUC 方差估计的偏差。
DOI：
10.1016/j.patrec.2023.12.012
复制
发表时间：
2024
影响因子：
5.1
通讯作者：
Xu,Jingyan
中科院分区：
文献类型：
--
作者：
Xu,Jingyan
受试者工作特征（ROC）曲线下面积（AUC）是量化和比较二元分类器的标准度量。估计AUC和相关变量（AUC的方差或多个相关AUC的完整协方差矩阵）的一种流行方法是DeLong等人（1988）提出的方法，该方法基于Mann Whitney双样本U统计量。方差估计量的偏差是假设检验和置信区间构建等应用中的一个重要因素-负偏差方差估计量可能导致不正确的结论，而正偏差是保守的，因此更可取。在这项工作中，我们表明，（协）方差估计在德隆的方法总是正偏置。更具体地，估计协方差的期望与真实协方差之间的差矩阵是半正定矩阵。当样本量较小时，该偏倚不可忽略，并随着样本量的增加而迅速减小。我们的方法依赖于从AUC内核构建一个随机变量，其（协）方差矩阵与偏差一致，从而建立索赔。我们还讨论了AUC方差估计的替代方法，可能会降低偏倚。
The area under the Receiver Operating Characteristic (ROC) curve (AUC) is a standard metric for quantifying and comparing binary classifiers. A popular approach to estimating the AUCs and the associated variabilities – the variance of the AUC or the full covariance matrix of multiple correlated AUCs – is the one proposed by DeLong et al. (1988), which is based on the Mann Whitney two-sample U-statistics. The bias of a variance estimator is an important factor in applications such as hypothesis testing and construction of confidence intervals – a negatively biased variance estimator may lead to incorrect conclusions, and a positive bias is conservative hence preferable. In this work, we show that the (co-)variance estimate in DeLong’s approach is always positively biased. More specifically, the difference matrix between the expectation of the estimated covariance and the true covariance is a positive semi-definite matrix. This bias is non-negligible when the sample size is small, and quickly diminishes as the sample size increases. Our method relies on constructing, from the AUC kernel, a random variable whose (co-)variance matrix coincides with the bias, thereby establishing the claim. We also discuss alternative approaches to AUC variance estimation that may potentially reduce the bias.