Understanding the Source of Semantic Regularities in Word Embeddings
Understanding the Source of Semantic Regularities in Word Embeddings
复制标题
了解词嵌入中语义规则的来源
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Z. Pardos
中科院分区：
文献类型：
--
作者：
Hsiao;José Camacho;Z. Pardos
语义关系是人类如何使用语言理解和表达真实的世界中的概念的核心。最近，有一个线程的研究，旨在通过学习向量表示从文本语料库中建模这些关系。这些方法中的大多数严格地集中在利用关系词对在句子中的共现。在本文中，我们调查的假设，语料库中的词汇关系的例子是神经词嵌入的能力，完成类比涉及的关系的基础。我们的实验中，我们从训练语料库中删除所有已知的关系的例子，只显示边缘退化的类比完成性能涉及删除的关系。这一发现增强了我们对神经词嵌入的理解，表明特定语义关系的共现信息不是其结构规律性的主要来源。
Semantic relations are core to how humans understand and express concepts in the real world using language. Recently, there has been a thread of research aimed at modeling these relations by learning vector representations from text corpora. Most of these approaches focus strictly on leveraging the co-occurrences of relationship word pairs within sentences. In this paper, we investigate the hypothesis that examples of a lexical relation in a corpus are fundamental to a neural word embedding’s ability to complete analogies involving the relation. Our experiments, in which we remove all known examples of a relation from training corpora, show only marginal degradation in analogy completion performance involving the removed relation. This finding enhances our understanding of neural word embeddings, showing that co-occurrence information of a particular semantic relation is the not the main source of their structural regularity.
DOI：
10.1162/tacl_a_00324
发表时间：
2020-01-01
影响因子：
10.9
作者：
Jiang, Zhengbao;Xu, Frank F.;Neubig, Graham
通讯作者：
Neubig, Graham