Online Platt Scaling with Calibeating
Online Platt Scaling with Calibeating
复制标题
DOI：
10.48550/arxiv.2305.00070
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Chirag Gupta;Aaditya Ramdas
中科院分区：
文献类型：
--
作者：
Chirag Gupta;Aaditya Ramdas
我们提出了一种在线事后校准方法，称为在线普拉特缩放（OPS），它结合了普拉特缩放技术与在线逻辑回归。我们证明，OPS顺利适应i.i.d.之间。和非I.I.D.具有分布漂移的设置。此外，在最好的普拉特缩放模型本身被误校准的情况下，我们通过将最近开发的技术称为校准跳动，使其更加强大，以提高OPS。从理论上讲，我们得到的OPS+校准方法可以保证针对对抗性结果序列进行校准。从经验上讲，它在一系列合成和真实世界的数据集上都是有效的，有或没有分布漂移，在没有超参数调整的情况下实现了上级性能。最后，我们将所有OPS思想扩展到Beta缩放方法。
We present an online post-hoc calibration method, called Online Platt Scaling (OPS), which combines the Platt scaling technique with online logistic regression. We demonstrate that OPS smoothly adapts between i.i.d. and non-i.i.d. settings with distribution drift. Further, in scenarios where the best Platt scaling model is itself miscalibrated, we enhance OPS by incorporating a recently developed technique called calibeating to make it more robust. Theoretically, our resulting OPS+calibeating method is guaranteed to be calibrated for adversarial outcome sequences. Empirically, it is effective on a range of synthetic and real-world datasets, with and without distribution drifts, achieving superior performance without hyperparameter tuning. Finally, we extend all OPS ideas to the beta scaling method.