The relative performance of ensemble methods with deep convolutional neural networks for image classification
The relative performance of ensemble methods with deep convolutional neural networks for image classification
复制标题
DOI：
10.1080/02664763.2018.1441383
复制
发表时间：
2018-01-01
影响因子：
1.5
通讯作者：
van der Laan, Mark
中科院分区：
文献类型：
--
作者：
Ju, Cheng;Bibaut, Aurelien;van der Laan, Mark
人工神经网络已经成功地应用于各种机器学习任务，包括图像识别、语义分割和机器翻译。然而，很少有研究充分研究人工神经网络的集成。在这项工作中，我们研究了多种广泛使用的集成方法，包括未加权平均、多数投票、贝叶斯最优分类器和(离散)超级学习器，并以深度神经网络为候选算法。我们设计了几个实验，候选算法是同一网络结构，在单一训练过程中具有不同的模型检查点，具有相同结构但随机多次训练的网络结构，以及具有不同结构的网络结构。此外，我们还进一步研究了神经网络的过度自信现象及其对集成方法的影响。在我们所有的实验中，超级学习者在这项研究的所有集成方法中取得了最好的性能。
Artificial neural networks have been successfully applied to a variety of machine learning tasks, including image recognition, semantic segmentation, and machine translation. However, few studies fully investigated ensembles of artificial neural networks. In this work, we investigated multiple widely used ensemble methods, including unweighted averaging, majority voting, the Bayes Optimal Classifier, and the (discrete) Super Learner, for image recognition tasks, with deep neural networks as candidate algorithms. We designed several experiments, with the candidate algorithms being the same network structure with different model checkpoints within a single training process, networks with same structure but trained multiple times stochastically, and networks with different structure. In addition, we further studied the overconfidence phenomenon of the neural networks, as well as its impact on the ensemble methods. Across all of our experiments, the Super Learner achieved best performance among all the ensemble methods in this study.