TCC-net: A two-stage training method with contradictory loss and co-teaching based on meta-learning for learning with noisy labels
TCC-net: A two-stage training method with contradictory loss and co-teaching based on meta-learning for learning with noisy labels
复制标题
DOI：
10.1016/j.ins.2023.119008
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Qiangqiang Xia;Feifei Lee;Qiu Chen
中科院分区：
文献类型：
--
作者：
Qiangqiang Xia;Feifei Lee;Qiu Chen
随着深度神经网络的快速发展，对大规模准确标记数据集的需求正在迅速增长。然而，人工标记的数据集往往会由于人工标记的错误而产生误导性信息。大多数以前的工作未能很好地控制训练模型对噪声标签的过拟合。在本文中，我们提出了一种新的两阶段学习框架，用于带噪声标签的学习，称为基于元学习的矛盾损失和协同教学的两阶段训练方法（TCC-net）。首先，设计了一种新的鲁棒损失函数--矛盾损失函数，用于预训练，实验结果和理论基础都证明了该函数具有足够的鲁棒性。在训练阶段，我们共同教导两个基于元学习的预训练网络，没有任何辅助的干净子集作为元数据。与其他协同教学方法不同，我们引入了两个多层感知器来帮助对所选样本进行加权，这意味着每个网络都通过其对等网络和自感知器使用加权选择的样本来更新自己。在Cifar 10、Cifar 100、Animal 10 N和Clothing 1 M等受损数据集上的实验结果表明，TCC-net在浅层上上级其他最先进的方法。具体来说，我们在具有80%对称噪声的合成Cifar 10上实现了12.59%的改进，在真实世界的Animal 10 N数据集上实现了0.27%的改进。代码发布于https://github.com/QiangqiangXia/TCC-net。
With the rapid development of deep neural networks, the demand for large-scale accurately labeled datasets is growing rapidly. However, human-labeled datasets often produce misleading information due to mistakes in manual labeling. Most previous works fail to control well the overfitting of the trained model to noisy labels. In this paper, we propose a novel two-stage learning framework for learning with noisy labels, called the Two-stage training method with Contradictory loss and Co-teaching based on meta-learning (TCC-net). First, a novel robust loss function called contradictory loss is designed for pre-training, which is proved to be sufficiently robust both in the experimental results and theoretical foundation. During the training stage, we co-teach two pretrained networks based on meta-learning without any auxiliary clean subset as meta-data. Unlike other co-teaching methods, we introduce two multilayer perceptron to assist in weighting the selected samples, meaning each network updates itself with weighted-selected samples by its peer network and self-perceptron. Experimental results on corrupted datasets, such as Cifar10, Cifar100, Animal10N, and Clothing1M, demonstrate that TCC-net is superior to other state-of-the-art methods on shallower layers. Specifically, we achieve 12.59% improvement on synthetic Cifar10 with 80% symmetric noise and 0.27% on the real-world Animal10N dataset. Code is released at https://github.com/QiangqiangXia/TCC-net.