What Don’t RNN Language Models Learn About Filler-Gap Dependencies?
What Don’t RNN Language Models Learn About Filler-Gap Dependencies?
复制标题
DOI：
10.7275/f7yj-1n62
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
R. Chaves
中科院分区：
文献类型：
--
作者：
R. Chaves
在一系列实验中，Wilcox等人（2018，2019b）提供了证据，表明通用的最先进的LSTM RNN语言模型不仅学会'约束（Ross，1967年）。在有关填充填充间隙构造基础的语言机制方面，有可能学到了数据集中的一些表面统计规律，而不是高级抽象的概括。
In a series of experiments Wilcox et al. (2018, 2019b) provide evidence suggesting that general-purpose state-of-the-art LSTM RNN language models have not only learned English filler-gap dependencies, but also some of their associated ‘island’ constraints (Ross, 1967)). In the present paper, I cast doubt on such claims, and argue that upon closer inspection filler-gap dependencies are learned only very imperfectly, including their associated island constraints. I conjecture that the LSTM RNN models in question have more likely learned some surface statistical regularities in the dataset rather than higher-level abstract generalizations about the linguistic mechanisms underlying filler-gap constructions.