Word Ordering as Unsupervised Learning Towards Syntactically Plausible Word Representations
Word Ordering as Unsupervised Learning Towards Syntactically Plausible Word Representations
复制标题
DOI：
--
复制
发表时间：
2017-11
期刊：
影响因子：
--
通讯作者：
Noriki Nishida;Hideki Nakayama
中科院分区：
文献类型：
--
作者：
Noriki Nishida;Hideki Nakayama
我们在这项研究中探索的研究问题是如何获得句法上合理的词表示，而不使用人类的注释。我们的基本假设是，单词排序测试，或线性化，是适合学习有关单词的句法知识。为了验证这一假设，我们开发了一个可区分的模型，称为单词排序网络（WON），显式地学习恢复正确的词序，同时隐式地获取表示句法知识的单词嵌入。我们评估所提出的方法产生的词嵌入下游的语法相关的任务，如词性标注和依赖分析。实验结果表明，在这些任务上，WON始终优于顺序不敏感和顺序敏感的基线。
The research question we explore in this study is how to obtain syntactically plausible word representations without using human annotations. Our underlying hypothesis is that word ordering tests, or linearizations, is suitable for learning syntactic knowledge about words. To verify this hypothesis, we develop a differentiable model called Word Ordering Network (WON) that explicitly learns to recover correct word order while implicitly acquiring word embeddings representing syntactic knowledge. We evaluate the word embeddings produced by the proposed method on downstream syntax-related tasks such as part-of-speech tagging and dependency parsing. The experimental results demonstrate that the WON consistently outperforms both order-insensitive and order-sensitive baselines on these tasks.