High-risk learning: acquiring new word vectors from tiny data
High-risk learning: acquiring new word vectors from tiny data
复制标题
高风险学习：从微小数据中获取新词向量
DOI：
10.18653/v1/d17-1030
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Marco Baroni
中科院分区：
文献类型：
--
作者：
Aurélie Herbelot;Marco Baroni
众所周知，分布式语义模型在处理小数据时会遇到困难。人们普遍认为，为了学习单词的“好向量”，模型必须有足够的用法示例。这与人类只能从少数几次出现中猜测单词含义的事实相矛盾。在本文中，我们表明，像Word 2 Vec这样的神经语言模型只需要对其标准架构进行微小的修改，就可以使用来自先前学习的语义空间的背景知识，从微小的数据中学习新术语。我们测试了我们的模型在单词定义和一个nonce任务，涉及2-6句话的上下文价值，表现出很大的提高，在国家的最先进的模型的定义任务。
Distributional semantics models are known to struggle with small data. It is generally accepted that in order to learn ‘a good vector’ for a word, a model must have sufficient examples of its usage. This contradicts the fact that humans can guess the meaning of a word from a few occurrences only. In this paper, we show that a neural language model such as Word2Vec only necessitates minor modifications to its standard architecture to learn new terms from tiny data, using background knowledge from a previously learnt semantic space. We test our model on word definitions and on a nonce task involving 2-6 sentences’ worth of context, showing a large increase in performance over state-of-the-art models on the definitional task.