Improving Word Representations via Global Context and Multiple Word Prototypes
Improving Word Representations via Global Context and Multiple Word Prototypes
复制标题
DOI：
--
复制
发表时间：
2012-07
期刊：
影响因子：
--
通讯作者：
E. Huang;R. Socher;Christopher D. Manning;A. Ng
中科院分区：
文献类型：
--
作者：
E. Huang;R. Socher;Christopher D. Manning;A. Ng
非监督单词表征在自然语言处理任务中非常有用，既可以作为学习算法的输入，也可以作为自然语言处理系统中的额外单词特征。然而，这些模型中的大多数都是在只有本地上下文和每个单词一个表示的情况下构建的。这是有问题的，因为单词通常是多义性的，而全球语境也可以为学习词义提供有用的信息。我们提出了一种新的神经网络体系结构，它1)通过结合局部和全局文档上下文来学习单词嵌入，从而更好地捕捉单词的语义；2)通过学习每个单词的多个嵌入来解释同音异义和多义性。我们引入了一个新的包含人类对句子语境中词对的判断的数据集，并在该数据集上对我们的模型进行了评估，结果表明我们的模型比竞争基线模型和其他神经语言模型的性能要好。
Unsupervised word representations are very useful in NLP tasks both as inputs to learning algorithms and as extra word features in NLP systems. However, most of these models are built with only local context and one representation per word. This is problematic because words are often polysemous and global context can also provide useful information for learning word meanings. We present a new neural network architecture which 1) learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and 2) accounts for homonymy and polysemy by learning multiple embeddings per word. We introduce a new dataset with human judgments on pairs of words in sentential context, and evaluate our model on it, showing that our model outperforms competitive baselines and other neural language models.