Are Prompt-based Models Clueless?
Are Prompt-based Models Clueless?
复制标题
基于提示的模型是否毫无头绪？
DOI：
10.48550/arxiv.2205.09295
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Yusuke Oda
中科院分区：
文献类型：
--
作者：
Pride Kavumba;Ryo Takahashi;Yusuke Oda
微调大型预先训练的语言模型与特定任务的头部已经推进了许多自然语言理解基准的最先进的技术。然而，具有特定任务头部的模型需要大量的训练数据，这使得它们容易受到学习和利用特定于数据集的表面线索的影响，而这些线索无法推广到其他数据集。提示通过重用语言模型头部和格式化任务输入以匹配预训练目标，减少了数据需求。因此，我们期望基于少量镜头提示的模型不会利用表面线索。本文提出了一个实证研究是否少数镜头提示为基础的模型也利用表面线索。在MNLI、SNLI、HANS和COPA上分析基于少量提示的模型发现，基于提示的模型也利用了表面线索。虽然模型在有肤浅线索的情况下表现良好，但在没有肤浅线索的情况下，它们的表现往往不如随机准确性，或者只是略微好于随机准确性。
Finetuning large pre-trained language models with a task-specific head has advanced the state-of-the-art on many natural language understanding benchmarks. However, models with a task-specific head require a lot of training data, making them susceptible to learning and exploiting dataset-specific superficial cues that do not generalize to other datasets.Prompting has reduced the data requirement by reusing the language model head and formatting the task input to match the pre-training objective. Therefore, it is expected that few-shot prompt-based models do not exploit superficial cues.This paper presents an empirical examination of whether few-shot prompt-based models also exploit superficial cues.Analyzing few-shot prompt-based models on MNLI, SNLI, HANS, and COPA has revealed that prompt-based models also exploit superficial cues. While the models perform well on instances with superficial cues, they often underperform or only marginally outperform random accuracy on instances without superficial cues.
DOI：
10.18653/v1/s18-1121
发表时间：
2018-06
期刊：
--
影响因子：
--
作者：
Ivan Habernal;Henning Wachsmuth;Iryna Gurevych;Benno Stein
通讯作者：
Ivan Habernal;Henning Wachsmuth;Iryna Gurevych;Benno Stein
DOI：
--
发表时间：
2020-02
期刊：
--
影响因子：
--
作者：
Ronan Le Bras;Swabha Swayamdipta;Chandra Bhagavatula;Rowan Zellers;Matthew E. Peters;Ashish Sabharwal;Yejin Choi
通讯作者：
Ronan Le Bras;Swabha Swayamdipta;Chandra Bhagavatula;Rowan Zellers;Matthew E. Peters;Ashish Sabharwal;Yejin Choi