What GPT Knows About Who is Who
What GPT Knows About Who is Who
复制标题
GPT 对谁是谁了解多少
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Christy Tanner
中科院分区：
文献类型：
--
作者：
Xiaohan Yang;Eduardo Peynetti;Vasco Meerman;Christy Tanner
共指消解是理解语篇和语言的一项关键任务，但大型语言模型(LLM)尚未带来广泛的好处。此外，共指关系解析系统在很大程度上依赖于监督标签，这是非常昂贵和难以注释的，因此使其适合于快速工程。在本文中，我们介绍了一种基于QA的即时工程方法，并识别了生成性的、预先训练的LLM在共指关系解析任务中的能力和局限性。我们的实验表明，GPT-2和GPT-Neo能够返回有效答案，但它们识别共指提及的能力有限且对提示敏感，导致结果不一致。
Coreference resolution – which is a crucial task for understanding discourse and language at large – has yet to witness widespread benefits from large language models (LLMs). Moreover, coreference resolution systems largely rely on supervised labels, which are highly expensive and difficult to annotate, thus making it ripe for prompt engineering. In this paper, we introduce a QA-based prompt-engineering method and discern generative, pre-trained LLMs’ abilities and limitations toward the task of coreference resolution. Our experiments show that GPT-2 and GPT-Neo can return valid answers, but that their capabilities to identify coreferent mentions are limited and prompt-sensitive, leading to inconsistent results.