A Unified Bias-Variance Decomposition for Zero-One and Squared Loss
A Unified Bias-Variance Decomposition for Zero-One and Squared Loss
复制标题
DOI：
--
复制
发表时间：
2000-07
期刊：
影响因子：
--
通讯作者：
Pedro M. Domingos
中科院分区：
文献类型：
--
作者：
Pedro M. Domingos
偏差方差分解是理解机器学习算法的一个非常有用且被广泛使用的工具。它最初是为平方损失开发的。近年来，有几位作者提出了分解零损失的方法，但每种方法都有明显的缺点。特别地，所有这些分解与原始的平方损失分解只有直观的关系。在本文中，我们定义了任意损失函数的偏差和方差，并表明所得到的分解专为平方损失情况下的标准分解，以及与Kong和Dietterich（1995）的零- 1情况的分解密切相关。同样的分解也适用于可变的错误分类成本。我们展示了统一定义的一些有趣的结果。例如，Schapire等人（1997）的“边际”概念可以表示为0 - 1偏差和方差的函数，从而可以将分类器集成的泛化误差与训练样本上的基本学习者的偏差和方差正式联系起来。统一定义下的实验可以带来进一步的认识。
The bias-variance decomposition is a very useful and widely-used tool for understanding machine-learning algorithms. It was originally developed for squared loss. In recent years, several authors have proposed decompositions for zero-one loss, but each has significant shortcomings. In particular, all of these decompositions have only an intuitive relationship to the original squared-loss one. In this paper, we define bias and variance for an arbitrary loss function, and show that the resulting decomposition specializes to the standard one for the squared-loss case, and to a close relative of Kong and Dietterich’ s (1995) one for the zero-one case. The same decomposition also applies to variable misclassification costs. We show a number of interesting consequences of the unified definition. For example, Schapire et al.’ s (1997) notion of “margin” can be expressed as a function of the zero-one bias and variance, making it possible to formally relate a classifier ensemble’s generalization error to the base learner’ s bias and variance on training examples. Experiments with the unified definition lead to further insights.