An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization
An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization
复制标题
DOI：
10.1023/a:1007607513941
复制
发表时间：
2000-08-01
期刊：
影响因子：
7.5
通讯作者：
Dietterich, TG
中科院分区：
文献类型：
--
作者：
Dietterich, TG
Bagging和boosting是通过操纵给定给“基础”学习算法的训练数据来生成不同分类器集合的方法。Breiman指出，它们的有效性依赖于基础学习算法的不稳定性。生成系综的另一种方法是随机化基本算法做出的内部决策。阿里和帕扎尼以及迪特里希和孔以前已经研究过这种一般方法。本文比较了随机化，装袋，提高决策树算法C4.5的性能的有效性。实验表明，在分类噪声很小或没有分类噪声的情况下，随机化与bagging有竞争力（可能略优于上级），但不如boosting准确。在有大量分类噪声的情况下，bagging比boosting好得多，有时比随机化好。
Bagging and boosting are methods that generate a diverse ensemble of classifiers by manipulating the training data given to a "base" learning algorithm. Breiman has pointed out that they rely for their effectiveness on the instability of the base learning algorithm. An alternative approach to generating an ensemble is to randomize the internal decisions made by the base algorithm. This general approach has been studied previously by Ali and Pazzani and by Dietterich and Kong. This paper compares the effectiveness of randomization, bagging, and boosting for improving the performance of the decision-tree algorithm C4.5. The experiments show that in situations with little or no classification noise, randomization is competitive with (and perhaps slightly superior to) bagging but not as accurate as boosting. In situations with substantial classification noise, bagging is much better than boosting, and sometimes better than randomization.