Visual Exploration of Semantic Relationships in Neural Word Embeddings
Visual Exploration of Semantic Relationships in Neural Word Embeddings
复制标题
DOI：
10.1109/tvcg.2017.2745141
复制
发表时间：
2018-01-01
影响因子：
5.2
通讯作者：
Pascucci, Valerio
中科院分区：
文献类型：
--
作者：
Liu, Shusen;Bremer, Peer-Timo;Pascucci, Valerio
通过神经语言模型构建单词的分布式表示并使用生成的向量空间进行分析已成为自然语言处理（NLP）的重要组成部分。然而，尽管它们的广泛应用，很少有人知道这些空间的结构和性质。为了深入了解单词之间的关系，NLP社区已经开始采用高维可视化技术。特别是，研究人员通常使用t分布随机邻居嵌入（t-SNE）和主成分分析（PCA）来创建二维嵌入，以评估整体结构和探索线性关系（例如，词的类比）。不幸的是，这些技术通常会产生平庸甚至误导性的结果，并且无法解决特定领域的可视化挑战，这些挑战对于理解单词嵌入中的语义关系至关重要。在这里，我们介绍了新的嵌入技术，可视化语义和句法类比，以及相应的测试，以确定所产生的意见是否捕捉到显着的结构。此外，我们介绍了两个新的观点，全面研究类比关系。最后，我们增加t-SNE嵌入来传达不确定性信息，以便进行可靠的解释。结合起来，不同的视图解决了一些难以用现有工具解决的特定于领域的任务。
Constructing distributed representations for words through neural language models and using the resulting vector spaces for analysis has become a crucial component of natural language processing (NLP). However, despite their widespread application, little is known about the structure and properties of these spaces. To gain insights into the relationship between words, the NLP community has begun to adapt high-dimensional visualization techniques. In particular, researchers commonly use t-distributed stochastic neighbor embeddings (t-SNE) and principal component analysis (PCA) to create two-dimensional embeddings for assessing the overall structure and exploring linear relationships (e.g., word analogies), respectively. Unfortunately, these techniques often produce mediocre or even misleading results and cannot address domain-specific visualization challenges that are crucial for understanding semantic relationships in word embeddings. Here, we introduce new embedding techniques for visualizing semantic and syntactic analogies, and the corresponding tests to determine whether the resulting views capture salient structures. Additionally, we introduce two novel views for a comprehensive study of analogy relationships. Finally, we augment t-SNE embeddings to convey uncertainty information in order to allow a reliable interpretation. Combined, the different views address a number of domain-specific tasks difficult to solve with existing tools.