How to initialize the CNN for small datasets: Extracting discriminative filters from pre-trained model
How to initialize the CNN for small datasets: Extracting discriminative filters from pre-trained model
复制标题
DOI：
10.1109/acpr.2015.7486549
复制
发表时间：
2015-11
期刊：
影响因子：
--
通讯作者：
Guanwen Zhang;Jien Kato;Yu Wang;K. Mase
中科院分区：
文献类型：
--
作者：
Guanwen Zhang;Jien Kato;Yu Wang;K. Mase
在本文中，我们研究如何初始化卷积神经网络（CNN）模型以在小数据集上进行训练。特别地，我们尝试从目标任务的预训练模型中提取判别过滤器。在相对熵和线性重建的基础上，提出了最小熵损失（MEL）和最小重建误差（MRE）两种方法。由所提出的 MEL 和 MRE 方法初始化的 CNN 模型能够快速收敛并实现更好的精度。我们在 CIFAR10、CIFAR100、SVHN 和 STL-10 公共数据集上评估 MEL 和 MRE。一致的性能证明了所提出方法的优点。
In this paper, we study how to initialize the convolutional neural network (CNN) model for training on a small dataset. Specially, we try to extract discriminative filters from the pre-trained model for a target task. On the basis of relative entropy and linear reconstruction, two methods, Minimum Entropy Loss (MEL) and Minimum Reconstruction Error (MRE), are proposed. The CNN models initialized by the proposed MEL and MRE methods are able to converge fast and achieve better accuracy. We evaluate MEL and MRE on the CIFAR10, CIFAR100, SVHN, and STL-10 public datasets. The consistent performances demonstrate the advantages of the proposed methods.