Learning with Instance-Dependent Label Noise: A Sample Sieve Approach
Learning with Instance-Dependent Label Noise: A Sample Sieve Approach
复制标题
DOI：
--
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Hao Cheng;Zhaowei Zhu;Xingyu Li;Yifei Gong;Xing Sun;Yang Liu
中科院分区：
文献类型：
--
作者：
Hao Cheng;Zhaowei Zhu;Xingyu Li;Yifei Gong;Xing Sun;Yang Liu
人工注释的标签通常容易产生噪声，而这种噪声的存在会降低生成的深度神经网络 (DNN) 模型的性能。大多数使用噪声标签学习的文献（除了最近的几个例外）都关注标签噪声独立于特征的情况。实际上，注释错误往往与实例相关，并且通常取决于识别特定任务的难度级别。应用独立于实例的设置的现有结果将需要大量的噪声率估计。因此，使用依赖于实例的标签噪声进行学习仍然是一个挑战。在本文中，我们提出了 CORES^2（COnfidence REgularized Sample Sieve），它逐步筛选出损坏的样本。 CORES^2 的实现不需要指定噪声率，但我们能够在过滤损坏的示例时提供 CORES^2 的理论保证。这种高质量的样本筛允许我们在训练 DNN 解决方案时分别处理干净的样本和损坏的样本，并且这种分离在依赖于实例的噪声设置中被证明是有利的。我们展示了 CORES^2 在具有合成实例相关标签噪声的 CIFAR10 和 CIFAR100 数据集以及具有真实世界人类噪声的 Clothing1M 上的性能。出于独立利益的考虑，我们的样本筛提供了一种用于解剖噪声数据集的通用机制，并为各种强大的训练技术提供了灵活的接口，以进一步提高性能。
Human-annotated labels are often prone to noise, and the presence of such noise will degrade the performance of the resulting deep neural network (DNN) models. Much of the literature (with several recent exceptions) of learning with noisy labels focuses on the case when the label noise is independent from features. Practically, annotations errors tend to be instance-dependent and often depend on the difficulty levels of recognizing a certain task. Applying existing results from instance-independent settings would require a significant amount of estimation of noise rates. Therefore, learning with instance-dependent label noise remains a challenge. In this paper, we propose CORES^2 (COnfidence REgularized Sample Sieve), which progressively sieves out corrupted samples. The implementation of CORES^2 does not require specifying noise rates and yet we are able to provide theoretical guarantees of CORES^2 in filtering out the corrupted examples. This high-quality sample sieve allows us to treat clean examples and the corrupted ones separately in training a DNN solution, and such a separation is shown to be advantageous in the instance-dependent noise setting. We demonstrate the performance of CORES^2 on CIFAR10 and CIFAR100 datasets with synthetic instance-dependent label noise and Clothing1M with real-world human noise. As of independent interests, our sample sieve provides a generic machinery for anatomizing noisy datasets and provides a flexible interface for various robust training techniques to further improve the performance.