Generative Topic Embedding: a Continuous Representation of Documents
Generative Topic Embedding: a Continuous Representation of Documents
复制标题
DOI：
10.18653/v1/p16-1063
复制
发表时间：
2016-06
期刊：
影响因子：
--
通讯作者：
Shaohua Li;Tat-Seng Chua;Jun Zhu;C. Miao
中科院分区：
文献类型：
--
作者：
Shaohua Li;Tat-Seng Chua;Jun Zhu;C. Miao
词嵌入通过在一个小的上下文窗口中利用词的局部搭配模式将词映射到一个低维的连续嵌入空间中。另一方面，主题建模通过利用同一文档中的全局词语搭配模式，将文档映射到低维主题空间。这两种模式是互补的。在本文中，我们提出了一个生成式的主题嵌入模型，以结合这两种类型的模式联合收割机。在我们的模型中，主题由嵌入向量表示，并在文档之间共享。每个单词出现的概率受其本地上下文和主题的影响。变分推理方法产生的主题嵌入以及主题混合比例为每个文档。它们共同表示低维连续空间中的文档。在两个文档分类任务中，我们的方法比现有的八种方法性能更好，功能更少。此外，我们用一个例子说明，我们的方法可以生成连贯的主题，即使是基于一个文档。
Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window. On the other hand, topic modeling maps documents onto a low-dimensional topic space, by utilizing the global word collocation patterns in the same document. These two types of patterns are complementary. In this paper, we propose a generative topic embedding model to combine the two types of patterns. In our model, topics are represented by embedding vectors, and are shared across documents. The probability of each word is influenced by both its local context and its topic. A variational inference method yields the topic embeddings as well as the topic mixing proportions for each document. Jointly they represent the document in a low-dimensional continuous space. In two document classification tasks, our method performs better than eight existing methods, with fewer features. In addition, we illustrate with an example that our method can generate coherent topics even based on only one document.