Precision-Recall-Gain Curves: PR Analysis Done Right
Precision-Recall-Gain Curves: PR Analysis Done Right
复制标题
DOI：
--
复制
发表时间：
2015-12
期刊：
影响因子：
--
通讯作者：
Peter A. Flach;Meelis Kull
中科院分区：
文献类型：
--
作者：
Peter A. Flach;Meelis Kull
查准率-查全率分析在二元分类的应用中大量存在，其中真阴性不增加价值，因此不应影响分类器性能的评估。也许是受试者工作特征（ROC）曲线和曲线下面积的许多优点的启发，许多研究人员已经采取了报告精度召回（PR）曲线和相关面积作为性能指标。我们在本文中证明，这种做法充满了困难，主要是因为不连贯的规模假设-例如，PR曲线下面积采用精度值的算术平均值，而Fβ评分采用调和平均值。我们展示了如何通过在不同的坐标系中绘制PR曲线来解决这个问题，并证明了新的精确-召回-增益曲线继承了ROC曲线的所有关键优点。特别地，在精确度-召回率-增益曲线下的区域传达谐波标度上的预期F1分数，并且精确度-召回率-增益曲线的凸形船体允许我们校准分类器的分数，以便针对凸形船体上的每个操作点确定该点优化Fβ的β值的区间。我们通过实验证明，传统PR曲线下的区域可以很容易地支持具有较低预期F1分数的模型，因此使用精确-召回-增益曲线将导致更好的模型选择。
Precision-Recall analysis abounds in applications of binary classification where true negatives do not add value and hence should not affect assessment of the classifier's performance. Perhaps inspired by the many advantages of receiver operating characteristic (ROC) curves and the area under such curves for accuracy-based performance assessment, many researchers have taken to report Precision-Recall (PR) curves and associated areas as performance metric. We demonstrate in this paper that this practice is fraught with difficulties, mainly because of incoherent scale assumptions - e.g., the area under a PR curve takes the arithmetic mean of precision values whereas the Fβ score applies the harmonic mean. We show how to fix this by plotting PR curves in a different coordinate system, and demonstrate that the new Precision-Recall-Gain curves inherit all key advantages of ROC curves. In particular, the area under Precision-Recall-Gain curves conveys an expected F1 score on a harmonic scale, and the convex hull of a Precision-Recall-Gain curve allows us to calibrate the classifier's scores so as to determine, for each operating point on the convex hull, the interval of β values for which the point optimises Fβ. We demonstrate experimentally that the area under traditional PR curves can easily favour models with lower expected F1 score than others, and so the use of Precision-Recall-Gain curves will result in better model selection.