Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models.
Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models.
复制标题
DOI：
10.1109/icdm.2016.0047
复制
发表时间：
2016-12
期刊：
影响因子：
--
通讯作者：
Cooper GF
中科院分区：
文献类型：
--
作者：
Naeini MP;Cooper GF
从数据中学习准确的概率模型对于数据挖掘的许多实际任务至关重要。在本文中，我们提出了一种新的非参数校准方法，称为近等渗回归集合（ENIR）。该方法可以被认为是最近提出的BBQ校准方法以及常用的基于等渗回归的校准方法（IsoRegC）的扩展。 ENIR 旨在解决 IsoRegC 的关键限制，即预测的单调性假设。与 BBQ 类似，该方法对二元分类器的输出进行后处理以获得校准概率。因此，它可以与许多现有的分类模型一起使用来生成准确的概率预测。我们展示了 ENIR 在常用二元分类模型的合成数据集和真实数据集上的性能。实验结果表明，该方法优于几种常见的二元分类器校准方法。特别是在真实数据上，ENIR 通常在统计上的表现明显优于其他方法，而且绝不会更差。它能够提高分类器的校准能力，同时保留其区分能力。该方法对于大规模数据集在计算上也很容易处理，因为它的时间为 O(N log N)，其中 N 是样本数。
Learning accurate probabilistic models from data is crucial in many practical tasks in data mining. In this paper we present a new non-parametric calibration method called ensemble of near isotonic regression (ENIR). The method can be considered as an extension of BBQ, a recently proposed calibration method, as well as the commonly used calibration method based on isotonic regression (IsoRegC). ENIR is designed to address the key limitation of IsoRegC which is the monotonicity assumption of the predictions. Similar to BBQ, the method post-processes the output of a binary classifier to obtain calibrated probabilities. Thus it can be used with many existing classification models to generate accurate probabilistic predictions. We demonstrate the performance of ENIR on synthetic and real datasets for commonly applied binary classification models. Experimental results show that the method outperforms several common binary classifier calibration methods. In particular on the real data, ENIR commonly performs statistically significantly better than the other methods, and never worse. It is able to improve the calibration power of classifiers, while retaining their discrimination power. The method is also computationally tractable for large scale datasets, as it is O(N log N) time, where N is the number of samples.