Supervised contrastive learning with corrected labels for noisy label learning
Supervised contrastive learning with corrected labels for noisy label learning
复制标题
DOI：
10.1007/s10489-023-05018-0
复制
发表时间：
2023-10
影响因子：
5.3
通讯作者：
Jihong Ouyang;Chenyang Lu;Bing Wang;C. Li
中科院分区：
文献类型：
--
作者：
Jihong Ouyang;Chenyang Lu;Bing Wang;C. Li
深度神经网络在人工智能领域和各种下游任务中取得了重大成功。它们将图像或文本编码成密集的特征表示，并由大量标记数据进行监督。由于高质量标签数据的昂贵，需要收集大量易于访问的实例进行有监督的学习。然而，它们没有经过专家的注释，因此可能包含大量噪声实例，这将降低性能。为了在误导噪声的情况下学习稳健的特征表示，我们使用有监督的对比学习来直接在隐藏空间中进行监督，而不是像流行的交叉熵损失函数那样在预测空间中进行监督。然而，有监督对比学习的前沿噪声标签学习方法往往会丢弃被认为有噪声的数据，从而不能容忍高比率噪声数据集。因此，我们提出了一种新的训练策略--带修正标签的监督对比学习(SCL)来抵抗噪声标签的攻击。SLL利用经验小损失假设对噪声标签进行校正，并使用校正后的数据进行有监督的对比学习。具体地说，我们使用生成的软标签作为监督信息，以便于我们实施监督对比学习。这种对比学习的扩展确保了监督信息的完整性，同时有效地加强了学习过程。此外，具有相同软标签的样本被视为正样本对，而具有不同软标签的样本被视为负样本对。在这种策略下，神经网络的表示将局部区分保持在一个小批次中。此外，我们还采用了原型对比学习技术来确保全局区分。我们的SLI在许多基准数据集上表现出了出色的性能，在各种标准化评估场景中展示了其有效性。此外，我们的模型在应用于真实世界的噪声数据集时被证明是非常有价值的。
Deep neural networks have achieved significant success in the artificial intelligence community and various downstream tasks. They encode images or texts into dense feature representations and are supervised by a large amount of labeled data. Due to the expensiveness of high-quality labeled data, a huge number of easy-to-access instances are collected to conduct supervised learning. However, they have not been annotated by experts and thus can contain numerous noisy instances, which will degrade the performance. To learn robust feature representations despite misleading noisy labels, we employ supervised contrastive learning to directly perform supervision in the hidden space, rather than in the prediction space like the prevalent cross-entropy loss function. However, cutting-edge noisy label learning methods with supervised contrastive learning always discard the data considered to be noisy, and thus cannot tolerate high-ratio noisy datasets. Therefore, we propose a novel training strategy named Supervised Contrastive Learning with Corrected Labels (Scl) to defend against the attack of noisy labels. Sclcorrects the noisy labels with an empirical small-loss assumption and conducts supervised contrastive learning using these corrected data. Specifically, we employ the generated soft labels as supervisory information to facilitate our implementation of supervised contrastive learning. This expansion of contrastive learning ensures the integrity of the supervisory information while effectively enhancing the learning process. In addition, samples sharing the same soft labels are treated as positive sample pairs, while those with different soft labels are considered to be negative sample pairs. With this strategy, the representations from neural networks keep the local discrimination in one mini-batch. Besides, we also employ a prototype contrastive learning technique to ensure global discrimination. Our Sclhas demonstrated excellent performance on numerous benchmark datasets, showcasing its effectiveness in various standardized evaluation scenarios. Additionally, our model has proven to be highly valuable when applied to real-world noisy datasets.