Pruning of Error Correcting Output Codes by optimization of accuracy-diversity trade off
Pruning of Error Correcting Output Codes by optimization of accuracy-diversity trade off
复制标题
DOI：
10.1007/s10994-014-5477-5
复制
发表时间：
2015-10-01
期刊：
影响因子：
7.5
通讯作者：
Smith, Raymond
中科院分区：
文献类型：
--
作者：
Ozogur-Akyuz, Sureyya;Windeatt, Terry;Smith, Raymond
集成学习是在有监督和无监督学习中结合学习者以获得更可靠和准确的预测的一种方法。然而，集合大小有时不必要地大，这会导致额外的内存使用、计算开销和降低的效率。为了克服这些副作用，已经开发了剪枝算法；因为这是一个组合问题，所以在计算上找不到准确的集合子集。不同类型的启发式算法被开发出来以获得近似解，但它们缺乏理论上的保证。纠错输出码(ECOC)是一种著名的多类分类集成技术，它结合二进制学习器的输出来预测多类数据的类。本文提出了一种同时利用精度和多样性信息对ECOC矩阵进行剪枝的新方法。现有的所有剪枝方法都需要集合的大小作为参数，因此剪枝方法的性能依赖于集合的大小。我们的非参数剪枝方法是新颖的，因为它不依赖于集合的大小。实验结果表明，我们的剪枝方法在很大程度上优于其他已有的方法。
Ensemble learning is a method of combining learners to obtain more reliable and accurate predictions in supervised and unsupervised learning. However, the ensemble sizes are sometimes unnecessarily large which leads to additional memory usage, computational overhead and decreased effectiveness. To overcome such side effects, pruning algorithms have been developed; since this is a combinatorial problem, finding the exact subset of ensembles is computationally infeasible. Different types of heuristic algorithms have developed to obtain an approximate solution but they lack a theoretical guarantee. Error Correcting Output Code (ECOC) is one of the well-known ensemble techniques for multiclass classification which combines the outputs of binary base learners to predict the classes for multiclass data. In this paper, we propose a novel approach for pruning the ECOC matrix by utilizing accuracy and diversity information simultaneously. All existing pruning methods need the size of the ensemble as a parameter, so the performance of the pruning methods depends on the size of the ensemble. Our unparametrized pruning method is novel as being independent of the size of ensemble. Experimental results show that our pruning method is mostly better than other existing approaches.