BOOSTING AND OTHER ENSEMBLE METHODS
BOOSTING AND OTHER ENSEMBLE METHODS
复制标题
DOI：
10.1162/neco.1994.6.6.1289
复制
发表时间：
1994-11-01
影响因子：
2.9
通讯作者：
VAPNIK, V
中科院分区：
文献类型：
--
作者：
DRUCKER, H;CORTES, C;VAPNIK, V
我们比较了三种类型的基于神经网络的集成技术的性能，一个单一的神经网络。集成算法是两个版本的boosting和委员会的神经网络的独立训练。对于四个算法中的每一个，我们实验确定的测试和训练误差曲线在光学字符识别（OCR）的问题作为训练集的大小和计算成本的函数，使用三种架构。我们表明，单个机器最适合小的训练集大小，而对于大的训练集大小，某些版本的boosting是最好的。然而，对于给定的计算成本，提升总是最好的。此外，我们为原始的boosting算法展示了一个令人惊讶的结果：即，随着训练集大小的增加，训练误差减小，直到它渐近于测试误差率。这在寻找更好的训练算法方面具有潜在的意义。
We compare the performance of three types of neural network-based ensemble techniques to that of a single neural network. The ensemble algorithms are two versions of boosting and committees of neural networks' trained independently. For each of the four algorithms, we experimentally determine the test and training error curves in an optical character recognition (OCR) problem as both a function of training set size and computational cost using three architectures. We show that a single machine is best for small training set size while for large training set size some version of boosting is best. However, for a given computational cost, boosting is always best. Furthermore, we show a surprising result for the original boosting algorithm: namely, that as the training set size increases, the training error decreases until it asymptotes to the test error rate. This has potential implications in the search for better training algorithms.