Exploring the Syntactic Abilities of RNNs with Multi-task Learning
Exploring the Syntactic Abilities of RNNs with Multi-task Learning
复制标题
通过多任务学习探索 RNN 的句法能力
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Tal Linzen
中科院分区：
文献类型：
--
作者：
Émile Enguehard;Yoav Goldberg;Tal Linzen
最近的研究利用主谓一致任务探索了rnn的句法能力，该任务可以诊断对句子结构的敏感性。rnn在普通情况下可以很好地完成这项任务，但在复杂句子中却表现不佳（Linzen et al., 2016）。我们测试了这些错误是由于体系结构的固有限制还是由于语料库中大多数协议依赖提供的相对间接的监督。我们训练了一个RNN来执行协议任务和一个额外的任务，要么是CCG超标记，要么是语言建模。多任务训练显著降低了错误率，特别是在复杂句子上，这表明rnn有能力进化出比以前更复杂的句法表示。我们还表明，容易获得的协议训练数据可以提高其他语法任务的性能，特别是当这些任务只有有限数量的训练数据可用时。多任务范式还可以用来将语法知识注入语言模型。
Recent work has explored the syntactic abilities of RNNs using the subject-verb agreement task, which diagnoses sensitivity to sentence structure. RNNs performed this task well in common cases, but faltered in complex sentences (Linzen et al., 2016). We test whether these errors are due to inherent limitations of the architecture or to the relatively indirect supervision provided by most agreement dependencies in a corpus. We trained a single RNN to perform both the agreement task and an additional task, either CCG supertagging or language modeling. Multi-task training led to significantly lower error rates, in particular on complex sentences, suggesting that RNNs have the ability to evolve more sophisticated syntactic representations than shown before. We also show that easily available agreement training data can improve performance on other syntactic tasks, in particular when only a limited amount of training data is available for those tasks. The multi-task paradigm can also be leveraged to inject grammatical knowledge into language models.
关于数字吸引力效应的解释：响应时间证据。
影响因子：
4.3
作者：
Staub,Adrian
通讯作者：
Staub,Adrian