DisturbLabel: Regularizing CNN on the Loss Layer
DisturbLabel: Regularizing CNN on the Loss Layer
复制标题
DOI：
10.1109/cvpr.2016.514
复制
发表时间：
2016-04
期刊：
影响因子：
--
通讯作者：
Lingxi Xie;Jingdong Wang;Zhen Wei;Meng Wang;Qi Tian
中科院分区：
文献类型：
--
作者：
Lingxi Xie;Jingdong Wang;Zhen Wei;Meng Wang;Qi Tian
在很长一段时间里，我们一直在用模型正则化来对抗CNN训练过程中的过拟合问题，包括权重衰减、模型平均、数据增强等。在本文中，我们提出了一个非常简单的算法，它在每次迭代中随机替换一部分标签为不正确的值。虽然故意生成不正确的训练标签看起来很奇怪，但我们表明，通过隐式平均许多使用不同标签集训练的网络，可以防止网络训练过度拟合。据我们所知，disrupblabel是第一个在损失层上添加噪声的作品。同时，与Dropout很好的配合，提供了互补的正则化功能。实验证明了在几种常用图像识别数据集上的竞争性识别结果。
During a long period of time we are combating overfitting in the CNN training process with model regularization, including weight decay, model averaging, data augmentation, etc. In this paper, we present DisturbLabel, an extremely simple algorithm which randomly replaces a part of labels as incorrect values in each iteration. Although it seems weird to intentionally generate incorrect training labels, we show that DisturbLabel prevents the network training from over-fitting by implicitly averaging over exponentially many networks which are trained with different label sets. To the best of our knowledge, DisturbLabel serves as the first work which adds noises on the loss layer. Meanwhile, DisturbLabel cooperates well with Dropout to provide complementary regularization functions. Experiments demonstrate competitive recognition results on several popular image recognition datasets.