CLUR: Uncertainty Estimation for Few-Shot Text Classification with Contrastive Learning
CLUR: Uncertainty Estimation for Few-Shot Text Classification with Contrastive Learning
复制标题
DOI：
10.1145/3580305.3599276
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
Jianfeng He;Xuchao Zhang;Shuo Lei;Abdulaziz Alhamadani;Fanglan Chen;Bei Xiao;Chang-Tien Lu
中科院分区：
文献类型：
--
作者：
Jianfeng He;Xuchao Zhang;Shuo Lei;Abdulaziz Alhamadani;Fanglan Chen;Bei Xiao;Chang-Tien Lu
小样本文本分类在样本采集昂贵或复杂的情况下有着广泛的应用。当分类错误的惩罚很高时，例如在数据稀缺的情况下进行早期威胁事件检测，我们希望知道“我们是否应该相信分类结果或重新检查它们。本文研究了少量文本分类的不确定性估计问题。给定有限的样本，UEFTC模型预测分类结果的不确定性得分，其是分类结果为假的可能性。然而，许多传统的文本分类不确定性估计模型不适合实现UEFTC模型。这些模型需要大量的训练样本，而UEFTC中的几个镜头设置只为一集中的每个类提供几个或一个支持样本。我们提出了从不确定性关系中进行对比学习（CLUR）来解决UEFTC问题。CLUR可以在伪不确定性分数的帮助下，对每个类只使用一个支持样本进行训练。与以往的作品，手动设置的伪不确定性分数，CLUR自适应学习他们使用我们提出的不确定性关系。具体来说，我们探讨了四个模型结构在CLUR调查的三个常用的对比学习组件在UEFTC的性能，并发现其中两个组件是有效的。实验结果表明，CLUR在4个数据集上的性能优于6个基线，其中在5路1次设置下，RCV 1数据集上的AUPR提高了4.52%。我们为UEFTC划分的代码和数据在https://github.com/he159ok/CLUR_UncertaintyEst_FewShot_TextCls。
Few-shot text classification has extensive application where the sample collection is expensive or complicated. When the penalty for classification errors is high, such as early threat event detection with scarce data, we expect to know "whether we should trust the classification results or reexamine them.'' This paper investigates the Uncertainty Estimation for Few-shot Text Classification (UEFTC), an unexplored research area. Given limited samples, a UEFTC model predicts an uncertainty score for a classification result, which is the likelihood that the classification result is false. However, many traditional uncertainty estimation models in text classification are unsuitable for implementing a UEFTC model. These models require numerous training samples, whereas the few-shot setting in UEFTC only provides a few or just one support sample for each class in an episode. We propose Contrastive Learning from Uncertainty Relations (CLUR) to address UEFTC. CLUR can be trained with only one support sample for each class with the help of pseudo uncertainty scores. Unlike previous works that manually set the pseudo uncertainty scores, CLUR self-adaptively learns them using our proposed uncertainty relations. Specifically, we explore four model structures in CLUR to investigate the performance of three common-used contrastive learning components in UEFTC and find that two of the components are effective. Experiment results prove that CLUR outperforms six baselines on four datasets, including an improvement of 4.52% AUPR on an RCV1 dataset in a 5-way 1-shot setting. Our code and data split for UEFTC are in https://github.com/he159ok/CLUR_UncertaintyEst_FewShot_TextCls.