Using AUC and accuracy in evaluating learning algorithms
Using AUC and accuracy in evaluating learning algorithms
复制标题
DOI：
10.1109/tkde.2005.50
复制
发表时间：
2005-03-01
影响因子：
8.9
通讯作者：
Ling, CX
中科院分区：
文献类型：
--
作者：
Huang, J;Ling, CX
自1970年代以来，ROC（接收器操作特征）曲线（接收器操作特征）曲线（接收器操作特征）曲线的区域一直在医学诊断中使用。最近已提出它是评估学习算法的预测能力的替代单数量度。但是，没有关于为什么应优先于准确性AUC的正式论点。在本文中，我们建立了对学习算法进行两种不同措施的正式标准，从理论和经验上讲，AUC比准确性更好（精确地定义）更好。然后，我们根据AUC的准确性重新评估了机器学习中公认的主张，并获得了有趣且令人惊讶的新结果。例如，已经建立了良好的公认，并接受了天真的贝叶斯和决策树的预测准确性非常相似。但是，我们表明，天真的贝叶斯比AUC的决策树要好得多。本文得出的结论可能会对机器学习和数据挖掘应用程序产生重大影响。
The area under the ROC ( Receiver Operating Characteristics) curve, or simply AUC, has been traditionally used in medical diagnosis since the 1970s. It has recently been proposed as an alternative single-number measure for evaluating the predictive ability of learning algorithms. However, no formal arguments were given as to why AUC should be preferred over accuracy. In this paper, we establish formal criteria for comparing two different measures for learning algorithms and we show theoretically and empirically that AUC is a better measure ( defined precisely) than accuracy. We then reevaluate well-established claims in machine learning based on accuracy using AUC and obtain interesting and surprising new results. For example, it has been well-established and accepted that Naive Bayes and decision trees are very similar in predictive accuracy. We show, however, that Naive Bayes is significantly better than decision trees in AUC. The conclusions drawn in this paper may make a significant impact on machine learning and data mining applications.