Solving Multiclass Learning Problems via Error-Correcting Output Codes
Solving Multiclass Learning Problems via Error-Correcting Output Codes
复制标题
DOI：
10.1613/jair.105
复制
发表时间：
1994-08
期刊：
影响因子：
--
通讯作者：
Thomas G. Dietterich;Ghulum Bakiri
中科院分区：
文献类型：
--
作者：
Thomas G. Dietterich;Ghulum Bakiri
多类学习问题涉及找到未知函数f（x）的定义，该未知函数f（x）的范围是包含k > 2个值的离散集（即，k“类”）。该定义是通过学习形式为（xi，f（xi））的训练示例集合获得的。现有的多类学习问题的方法包括直接应用多类算法，如决策树算法C4.5和CART，应用二进制概念学习算法来学习每个k类的单独二进制函数，以及应用具有分布式输出表示的二进制概念学习算法。本文比较了这三种方法的一种新技术，其中纠错码采用分布式输出表示。我们表明，这些输出表示提高了C4.5和反向传播在广泛的多类学习任务的泛化性能。我们还证明，这种方法是强大的训练样本的大小变化，分配的分布式表示到特定的类，和应用过拟合避免技术，如决策树修剪。最后，我们表明--与其他方法一样--纠错码技术可以提供可靠的类概率估计。两者合计，这些结果表明，纠错输出代码提供了一个通用的方法，提高性能的归纳学习程序的多类问题。
Multiclass learning problems involve finding a definition for an unknown function f(x) whose range is a discrete set containing k > 2 values (i.e., k "classes"). The definition is acquired by studying collections of training examples of the form (xi, f(xi)). Existing approaches to multiclass learning problems include direct application of multiclass algorithms such as the decision-tree algorithms C4.5 and CART, application of binary concept learning algorithms to learn individual binary functions for each of the k classes, and application of binary concept learning algorithms with distributed output representations. This paper compares these three approaches to a new technique in which error-correcting codes are employed as a distributed output representation. We show that these output representations improve the generalization performance of both C4.5 and backpropagation on a wide range of multiclass learning tasks. We also demonstrate that this approach is robust with respect to changes in the size of the training sample, the assignment of distributed representations to particular classes, and the application of overfitting avoidance techniques such as decision-tree pruning. Finally, we show that--like the other methods--the error-correcting code technique can provide reliable class probability estimates. Taken together, these results demonstrate that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems.