Flexible Generation from Fragmentary Linguistic Input
Flexible Generation from Fragmentary Linguistic Input
复制标题
DOI：
10.18653/v1/2022.acl-long.563
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Peng Qian;R. Levy
中科院分区：
文献类型：
--
作者：
Peng Qian;R. Levy
当今新型NLP任务中高性能模型的主要范式是通过从头开始训练或微调大型预训练的大型预培训模型的直接专业化。但是，直接专业化是否会捕获人类如何处理新的语言任务？我们假设人类表现的特征是通过构成人类语言用户可获得的基本计算图案的灵活推断。为了检验这一假设，我们制定了一组新型的零碎文本完成任务，并将三个直接特殊模型的行为与我们介绍的新模型GibbsComplete进行比较，该模型构成了两个基本的计算基础，这是当代模型的中心：掩盖和自动性单词预言。我们进行了三种类型的评估：人类对完成质量的判断，对输入片段施加的句法约束的满意度以及在完成结构统计中与人类行为的相似性。在没有任务特定的参数调整的情况下，GibbsComplete在前两个评估中的表现与直接特殊化模型相当，并且在第三次评估中胜过所有直接特殊模型。这些结果支持我们的假设，即新型语言任务和环境中的人类行为可以更好地以基本计算图案的灵活组成而不是直接专业化为特征。
The dominant paradigm for high-performance models in novel NLP tasks today is direct specialization for the task via training from scratch or fine-tuning large pre-trained models. But does direct specialization capture how humans approach novel language tasks? We hypothesize that human performance is better characterized by flexible inference through composition of basic computational motifs available to the human language user. To test this hypothesis, we formulate a set of novel fragmentary text completion tasks, and compare the behavior of three direct-specialization models against a new model we introduce, GibbsComplete, which composes two basic computational motifs central to contemporary models: masked and autoregressive word prediction. We conduct three types of evaluation: human judgments of completion quality, satisfaction of syntactic constraints imposed by the input fragment, and similarity to human behavior in the structural statistics of the completions. With no task-specific parameter tuning, GibbsComplete performs comparably to direct-specialization models in the first two evaluations, and outperforms all direct-specialization models in the third evaluation. These results support our hypothesis that human behavior in novel language tasks and environments may be better characterized by flexible composition of basic computational motifs rather than by direct specialization.