What do RNN Language Models Learn about Filler–Gap Dependencies?
What do RNN Language Models Learn about Filler–Gap Dependencies?
复制标题
RNN 语言模型如何了解填充间隙依赖关系？
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Richard Futrell
中科院分区：
文献类型：
--
作者：
Ethan Gotlieb Wilcox;R. Levy;Takashi Morita;Richard Futrell
RNN语言模型已经取得了最先进的困惑结果，并已被证明在一系列NLP任务中是有用的，但目前还不清楚它们学习了什么语法概括。在这里，我们研究最先进的RNN语言模型是否代表长距离的填充缺口依赖关系和对它们的约束。在实验控制的句子上检查RNN行为，旨在暴露填充缺口依赖关系，我们表明RNN可以在多个句法位置和大跨度的文本中表示关系。此外，我们还证明了RNN学习了已知的填充缺口依赖性限制的一个子集，称为岛约束：RNN显示了wh岛，附属岛和复杂NP岛的证据。这些研究表明，最先进的RNN模型能够学习和概括空句法位置。
RNN language models have achieved state-of-the-art perplexity results and have proven useful in a suite of NLP tasks, but it is as yet unclear what syntactic generalizations they learn. Here we investigate whether state-of-the-art RNN language models represent long-distance filler–gap dependencies and constraints on them. Examining RNN behavior on experimentally controlled sentences designed to expose filler–gap dependencies, we show that RNNs can represent the relationship in multiple syntactic positions and over large spans of text. Furthermore, we show that RNNs learn a subset of the known restrictions on filler–gap dependencies, known as island constraints: RNNs show evidence for wh-islands, adjunct islands, and complex NP islands. These studies demonstrates that state-of-the-art RNN models are able to learn and generalize about empty syntactic positions.
DOI：
10.1037/h0087426
发表时间：
2003-09-01
影响因子：
1.3
作者：
Masson, MEJ;Loftus, GR
通讯作者：
Loftus, GR
影响因子：
4.3
作者：
Barr, Dale J.;Levy, Roger;Scheepers, Christoph;Tily, Harry J.
通讯作者：
Tily, Harry J.