Neighborhood-Regularized Self-Training for Learning with Few Labels
Neighborhood-Regularized Self-Training for Learning with Few Labels
复制标题
DOI：
10.48550/arxiv.2301.03726
复制
发表时间：
2023-01
期刊：
影响因子：
--
通讯作者：
Ran Xu;Yue Yu;Hejie Cui;Xuan Kan;Yanqiao Zhu;Joyce Ho;Chao Zhang;Carl Yang
中科院分区：
文献类型：
--
作者：
Ran Xu;Yue Yu;Hejie Cui;Xuan Kan;Yanqiao Zhu;Joyce Ho;Chao Zhang;Carl Yang
有限监督训练深度神经网络(DNN)可以显著减轻标注负担，因此一直是一个热门的研究课题。自训练已成功地应用于半监督学习任务中，但它的一个缺点是容易受到错误伪标签带来的标签噪声的影响。受具有相似标签的样本往往具有相似表示的事实的启发，我们提出了一种基于邻域的样本选择方法来解决噪声伪标签的问题。我们通过在样本选择过程中聚合不同回合的预测来进一步稳定自我训练。在8个任务上的实验表明，我们提出的方法在文本和图形数据集上的性能平均提高了1.83%和2.51%，超过了最强的自我训练基线。进一步的分析表明，与最佳基线相比，我们提出的数据选择策略降低了36.8%的伪标签噪声，节省了57.3%的时间。我们的代码和附录将上载到https://github.com/ritaranx/NeST.
Training deep neural networks (DNNs) with limited supervision has been a popular research topic as it can significantly alleviate the annotation burden. Self-training has been successfully applied in semi-supervised learning tasks, but one drawback of self-training is that it is vulnerable to the label noise from incorrect pseudo labels. Inspired by the fact that samples with similar labels tend to share similar representations, we develop a neighborhood-based sample selection approach to tackle the issue of noisy pseudo labels. We further stabilize self-training via aggregating the predictions from different rounds during sample selection. Experiments on eight tasks show that our proposed method outperforms the strongest self-training baseline with 1.83% and 2.51% performance gain for text and graph datasets on average. Our further analysis demonstrates that our proposed data selection strategy reduces the noise of pseudo labels by 36.8% and saves 57.3% of the time when compared with the best baseline. Our code and appendices will be uploaded to https://github.com/ritaranx/NeST.