Accurate ensemble pruning with PL-bagging
Accurate ensemble pruning with PL-bagging
复制标题
DOI：
10.1016/j.csda.2014.09.003
复制
发表时间：
2015-03
期刊：
影响因子：
--
通讯作者：
Dongjun Chung;Hyunjoong Kim
中科院分区：
文献类型：
--
作者：
Dongjun Chung;Hyunjoong Kim
集成剪枝处理在组合之前选择基学习器，以提高预测准确性和效率。在集成文献中，已经指出，为了使集成分类器获得更高的预测精度，集成分类器由准确的分类器组成，同时尽可能多样化的分类器至关重要。在本文中，提出了一种新的集成剪枝方法，称为 PL-bagging。为了实现基学习器的多样性和准确性之间的平衡，PL-bagging 在组合步骤中采用正 Lasso 为基学习器分配权重。模拟研究和理论研究表明，PL-bagging 过滤掉了冗余的基础学习器，同时为更准确的基础学习器分配了更高的权重。这种改进的 PL-bagging 加权方案进一步提高了分类精度，并且随着集成规模的增加，这种改进变得更加显着。使用 22 个真实数据集和 4 个合成数据集，将 PL-bagging 的性能与用于聚合引导基础学习器的最先进的集成剪枝方法进行了比较。结果表明，PL-bagging 明显优于最先进的集成剪枝方法，例如基于 Boosting 的剪枝和 Trimmed bagging。
Ensemble pruning deals with the selection of base learners prior to combination in order to improve prediction accuracy and efficiency. In the ensemble literature, it has been pointed out that in order for an ensemble classifier to achieve higher prediction accuracy, it is critical for the ensemble classifier to consist of accurate classifiers which at the same time diverse as much as possible. In this paper, a novel ensemble pruning method, called PL-bagging, is proposed. In order to attain the balance between diversity and accuracy of base learners, PL-bagging employs positive Lasso to assign weights to base learners in the combination step. Simulation studies and theoretical investigation showed that PL-bagging filters out redundant base learners while it assigns higher weights to more accurate base learners. Such improved weighting scheme of PL-bagging further results in higher classification accuracy and the improvement becomes even more significant as the ensemble size increases. The performance of PL-bagging was compared with state-of-the-art ensemble pruning methods for aggregation of bootstrapped base learners using 22 real and 4 synthetic datasets. The results indicate that PL-bagging significantly outperforms state-of-the-art ensemble pruning methods such as Boosting-based pruning and Trimmed bagging.