Top-label calibration and multiclass-to-binary reductions
Top-label calibration and multiclass-to-binary reductions
复制标题
DOI：
--
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Chirag Gupta;Aaditya Ramdas
中科院分区：
文献类型：
--
作者：
Chirag Gupta;Aaditya Ramdas
如果预测类别-顶部标签-的报告概率被校准，并以顶部标签为条件，则多类别fier被称为顶部标签校准。这种对顶级标签的制约来自于一个密切相关的流行概念--连续校准，我们认为这使得连续校准和fi崇拜可以为决策做出解释。我们提出了顶标校准作为连续校准的一种直接fi连接。此外，我们概述了一个多类到二进制(M2B)归约框架，该框架包括统一的fifi估计、顶层标记和分类校准等。顾名思义，M2B的工作原理是将多类校准减少为大量的二进制校准问题，每个问题都可以使用简单的二进制校准例程来解决。我们使用研究良好的直方图库(HB)二值校准器来实例化M2B框架，并且证明了整个过程是多类校准的，而不需要对底层数据分布做任何假设。在对CIFAR-10和CIFAR-100上的四个深层网络结构进行的经验评估中，我们fi发现，M2B+HB方法比其他方法(如温度标度)获得更低的顶标和分类校准误差。这项工作的代码可以在https://github.com/aigen/df-posthoc-calibration上找到。
A multiclass classifier is said to be top-label calibrated if the reported probability for the predicted class—the top-label—is calibrated, conditioned on the top-label. This conditioning on the top-label is ab-sent in the closely related and popular notion of confidence calibration, which we argue makes confidence calibration difficult to interpret for decision-making. We propose top-label calibration as a rectification of confidence calibration. Further, we outline a multiclass-to-binary (M2B) reduction framework that unifies confidence, top-label, and class-wise calibration, among others. As its name suggests, M2B works by reducing multiclass calibration to numerous binary calibration problems, each of which can be solved using simple binary calibration routines. We instantiate the M2B framework with the well-studied histogram binning (HB) binary calibrator, and prove that the overall procedure is multiclass calibrated without making any assumptions on the underlying data distribution. In an empirical evaluation with four deep net architectures on CIFAR-10 and CIFAR-100, we find that the M2B + HB procedure achieves lower top-label and class-wise calibration error than other approaches such as temperature scaling. Code for this work is available at https://github.com/aigen/df-posthoc-calibration .