Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models
Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models
复制标题
使用启动来揭示神经语言模型中句法表示的组织
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Tal Linzen
中科院分区：
文献类型：
--
作者：
Grusha Prasad;Marten van Schijndel;Tal Linzen
神经语言模型（LM）在需要对句法结构敏感的任务上表现良好。借鉴心理语言学的句法启动范式，我们提出了一种新的技术来分析表征，使这种成功。通过建立结构之间的梯度相似性度量，该技术允许我们重建LM的句法表征空间的组织。我们使用这种技术来证明LSTM LM对不同类型的带有关系从句的句子的表示是以语言可解释的方式分层组织的，这表明LM跟踪句子的抽象属性。
Neural language models (LMs) perform well on tasks that require sensitivity to syntactic structure. Drawing on the syntactic priming paradigm from psycholinguistics, we propose a novel technique to analyze the representations that enable such success. By establishing a gradient similarity metric between structures, this technique allows us to reconstruct the organization of the LMs’ syntactic representational space. We use this technique to demonstrate that LSTM LMs’ representations of different types of sentences with relative clauses are organized hierarchically in a linguistically interpretable manner, suggesting that the LMs track abstract properties of the sentence.
影响因子：
5.4
作者：
Chang, F;Dell, GS;Bock, K
通讯作者：
Bock, K