Retraining: A Simple Way to Improve the Ensemble Accuracy of Deep Neural Networks for Image Classification
Retraining: A Simple Way to Improve the Ensemble Accuracy of Deep Neural Networks for Image Classification
复制标题
再训练：提高图像分类深度神经网络集成精度的简单方法
DOI：
10.1109/icpr.2018.8545535
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Einoshin Suzuki
中科院分区：
文献类型：
--
作者：
Kaikai Zhao;Tetsu Matsukawa;Einoshin Suzuki
在本文中，我们提出了一种新的启发式训练方法来帮助深度神经网络（DNN）反复摆脱局部最小值并移动到更好的局部最小值。我们的方法多次重复以下过程：随机重新初始化收敛DNN的最后一层的权值，同时保留其余层的权值，然后进行新一轮的训练。其动机是使新一轮的训练在前一轮学习到的“好的”初始参数的基础上学习到更好的参数。使用基于我们的训练过程训练的多个随机初始化dnn，我们可以获得比常规训练过程更准确和多样化的dnn集合。我们称这个框架为“再培训”。在八个深度神经网络模型上的实验表明，我们的方法总体上优于最先进的集成学习方法。我们还提供了再训练框架的两种变体来解决集成学习的任务，其中1)dnn具有非常高的训练精度（例如）和2)dnn的计算成本太高而无法训练。
In this paper, we propose a new heuristic training procedure to help a deep neural network (DNN) repeatedly escape from a local minimum and move to a better local minimum. Our method repeats the following processes multiple times: randomly reinitializing the weights of the last layer of a converged DNN while preserving the weights of the remaining layers, and then conducting a new round of training. The motivation is to make the training in the new round learn better parameters based on the “good” initial parameters learned in the previous round. With multiple randomly initialized DNNs trained based on our training procedure, we can obtain an ensemble of DNNs that are more accurate and diverse compared with the normal training procedure. We call this framework “retraining”. Experiments on eight DNN models show that our method generally outperforms the state-of-the-art ensemble learning methods. We also provide two variants of the retraining framework to tackle the tasks of ensemble learning in which 1) DNNs exhibit very high training accuracies (e.g., ) and 2) DNNs are too computationally expensive to train.