MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
复制标题
DOI：
--
复制
发表时间：
2017-12
期刊：
影响因子：
--
通讯作者：
Lu Jiang;Zhengyuan Zhou;Thomas Leung;Li-Jia Li-Li-Jia-Li-2040091191;Li Fei-Fei-Li-Fei-Fei-48004138
中科院分区：
文献类型：
--
作者：
Lu Jiang;Zhengyuan Zhou;Thomas Leung;Li-Jia Li-Li-Jia-Li-2040091191;Li Fei-Fei-Li-Fei-Fei-48004138
即使标签完全随机，最近的深度网络也能够记住整个数据。为了克服损坏标签上的过拟合，我们提出了一种学习另一种神经网络的新技术，称为MentorNet，以监督基础深度网络（即StudentNet）的训练。在培训过程中，MentorNet为StudentNet提供课程（样本加权方案），以专注于标签可能正确的样本。与通常由人类专家预先定义的现有课程不同，MentorNet使用StudentNet动态学习数据驱动的课程。实验结果表明，该方法能够显著提高在受损训练数据上训练的深度网络的泛化性能。值得注意的是，据我们所知，我们在WebVision上获得了最好的发布结果，这是一个包含220万张真实世界噪声标签图像的大型基准测试。代码在这个https URL
Recent deep networks are capable of memorizing the entire data even when the labels are completely random. To overcome the overfitting on corrupted labels, we propose a novel technique of learning another neural network, called MentorNet, to supervise the training of the base deep networks, namely, StudentNet. During training, MentorNet provides a curriculum (sample weighting scheme) for StudentNet to focus on the sample the label of which is probably correct. Unlike the existing curriculum that is usually predefined by human experts, MentorNet learns a data-driven curriculum dynamically with StudentNet. Experimental results demonstrate that our approach can significantly improve the generalization performance of deep networks trained on corrupted training data. Notably, to the best of our knowledge, we achieve the best-published result on WebVision, a large benchmark containing 2.2 million images of real-world noisy labels. The code are at this https URL