Learning Deep Neural Networks under Agnostic Corrupted Supervision
Learning Deep Neural Networks under Agnostic Corrupted Supervision
复制标题
DOI：
--
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Boyang Liu;Mengying Sun;Ding Wang;P. Tan;Jiayu Zhou
中科院分区：
文献类型：
--
作者：
Boyang Liu;Mengying Sun;Ding Wang;P. Tan;Jiayu Zhou
在监督受到破坏的情况下训练深层神经模型是具有挑战性的，因为被破坏的数据点可能会显著影响泛化性能。为了缓解这一问题，我们提出了一种高效的稳健算法，该算法在不假设损坏类型的情况下实现了强大的保证，并为分类和回归问题提供了一个统一的框架。不同于许多现有方法量化数据点的质量(例如，基于数据点的单个损失值)并相应地对其进行过滤，所提出的算法侧重于控制数据点对平均梯度的集体影响。即使我们的算法未能排除损坏的数据点，与基于损失值的最新过滤方法相比，该数据点对总体损失的影响也非常有限。在多个基准数据集上的大量实验证明了该算法在不同类型的损坏情况下的健壮性。
Training deep neural models in the presence of corrupted supervision is challenging as the corrupted data points may significantly impact the generalization performance. To alleviate this problem, we present an efficient robust algorithm that achieves strong guarantees without any assumption on the type of corruption, and provides a unified framework for both classification and regression problems. Unlike many existing approaches that quantify the quality of the data points (e.g., based on their individual loss values), and filter them accordingly, the proposed algorithm focuses on controlling the collective impact of data points on the average gradient. Even when a corrupted data point failed to be excluded by our algorithm, the data point will have very limited impact on the overall loss, as compared with state-of-the-art filtering methods based on loss values. Extensive experiments on multiple benchmark datasets have demonstrated the robustness of our algorithm under different types of corruptions.