AUC:: a misleading measure of the performance of predictive distribution models
AUC:: a misleading measure of the performance of predictive distribution models
复制标题
DOI：
10.1111/j.1466-8238.2007.00358.x
复制
发表时间：
2008-03-01
影响因子：
6.4
通讯作者：
Real, Raimundo
中科院分区：
文献类型：
--
作者：
Lobo, Jorge M.;Jimenez-Valverde, Alberto;Real, Raimundo
受试者工作特征（ROC）曲线下面积，称为AUC，目前被认为是评估预测分布模型准确性的标准方法。它避免了假设的主观性阈值选择过程中，连续概率导出的分数转换为一个二元存在-不存在变量，通过总结所有可能的阈值的整体模型性能。在这篇手稿中，我们回顾了这种措施的一些特点，并提出质疑，其可靠性作为一个比较措施的准确性模型的结果。我们不建议使用AUC，原因有五个：（1）它忽略了预测的概率值和模型的拟合优度;（2）它总结了ROC空间中很少操作的区域的测试性能;（3）它对遗漏和委托错误进行了同等权重;（4）它没有给出关于模型错误的空间分布的信息;（5）它没有给出关于模型错误的空间分布的信息。最重要的是，（5）模型执行的总体程度高度影响预测良好的缺勤率和AUC评分。
The area under the receiver operating characteristic (ROC) curve, known as the AUC, is currently considered to be the standard method to assess the accuracy of predictive distribution models. It avoids the supposed subjectivity in the threshold selection process, when continuous probability derived scores are converted to a binary presence-absence variable, by summarizing overall model performance over all possible thresholds. In this manuscript we review some of the features of this measure and bring into question its reliability as a comparative measure of accuracy between model results. We do not recommend using AUC for five reasons: (1) it ignores the predicted probability values and the goodness-of-fit of the model; (2) it summarises the test performance over regions of the ROC space in which one would rarely operate; (3) it weights omission and commission errors equally; (4) it does not give information about the spatial distribution of model errors; and, most importantly, (5) the total extent to which models are carried out highly influences the rate of well-predicted absences and the AUC scores.