Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs
Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs
复制标题
DOI：
10.18653/v1/2020.acl-main.303
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Michael A. Lepori;Tal Linzen;R. Thomas McCoy
中科院分区：
文献类型：
--
作者：
Michael A. Lepori;Tal Linzen;R. Thomas McCoy
基于序列的神经网络对句法结构表现出显著的敏感性，但它们在句法任务上的表现仍然不如基于树的网络。这种基于树的网络可以被提供有选区解析、依赖性解析或两者。我们评估这两个代表性的计划更有效地介绍了偏见的句法结构，提高性能的主谓一致预测任务。我们发现，一个基于成分的网络比一个基于依赖关系的网络更强大的概括，并结合这两种类型的结构不会产生进一步的改善。最后，我们表明，顺序模型的语法稳健性可以大大提高微调少量的构造数据，这表明数据增强是一个可行的替代显式选区结构给予顺序模型缺乏的语法偏见。
Sequence-based neural networks show significant sensitivity to syntactic structure, but they still perform less well on syntactic tasks than tree-based networks. Such tree-based networks can be provided with a constituency parse, a dependency parse, or both. We evaluate which of these two representational schemes more effectively introduces biases for syntactic structure that increase performance on the subject-verb agreement prediction task. We find that a constituency-based network generalizes more robustly than a dependency-based one, and that combining the two types of structure does not yield further improvement. Finally, we show that the syntactic robustness of sequential models can be substantially improved by fine-tuning on a small amount of constructed data, suggesting that data augmentation is a viable alternative to explicit constituency structure for imparting the syntactic biases that sequential models are lacking.