The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation
The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation
复制标题
DOI：
10.1186/s12864-019-6413-7
复制
发表时间：
2020-01-02
期刊：
影响因子：
4.4
通讯作者：
Jurman, Giuseppe
中科院分区：
文献类型：
--
作者：
Chicco, Davide;Jurman, Giuseppe
为了评估二元分类及其混淆矩阵，科学研究人员可以根据他们正在研究的实验目标采用几个统计率。尽管这是机器学习中的一个关键问题，但尚未就统一的选择性措施达成广泛共识。在混淆矩阵上计算的准确度和F-1得分一直是（现在仍然是）二进制分类任务中最流行的指标之一。然而，这些统计指标可能会危险地显示出过于乐观的夸大结果，特别是在不平衡的datasets.ResultsThe马修斯相关系数（MCC），而不是，是一个更可靠的统计率，只有当预测在所有四个混淆矩阵类别中获得良好的结果时，才会产生高分（真阳性、假阴性、真阴性和假阳性），与数据集中阳性元素的大小和阴性元素的大小成比例。结论在本文中，我们展示了MCC如何在评估二进制分类时产生比准确性和F-1分数更有信息量和真实性的分数，首先解释了MCC的数学属性，然后在六个合成用例和真实的基因组学场景中说明了MCC的资产。我们认为，马修斯相关系数应首选的准确性和F-1分数在评价二元分类任务的所有科学界。
BackgroundTo evaluate binary classifications and their confusion matrices, scientific researchers can employ several statistical rates, accordingly to the goal of the experiment they are investigating. Despite being a crucial issue in machine learning, no widespread consensus has been reached on a unified elective chosen measure yet. Accuracy and F-1 score computed on confusion matrices have been (and still are) among the most popular adopted metrics in binary classification tasks. However, these statistical measures can dangerously show overoptimistic inflated results, especially on imbalanced datasets.ResultsThe Matthews correlation coefficient (MCC), instead, is a more reliable statistical rate which produces a high score only if the prediction obtained good results in all of the four confusion matrix categories (true positives, false negatives, true negatives, and false positives), proportionally both to the size of positive elements and the size of negative elements in the dataset.ConclusionsIn this article, we show how MCC produces a more informative and truthful score in evaluating binary classifications than accuracy and F-1 score, by first explaining the mathematical properties, and then the asset of MCC in six synthetic use cases and in a real genomics scenario. We believe that the Matthews correlation coefficient should be preferred to accuracy and F-1 score in evaluating binary classification tasks by all scientific communities.