Mini-Batch Variational Inference for Time-Aware Topic Modeling
Mini-Batch Variational Inference for Time-Aware Topic Modeling
复制标题
DOI：
10.1007/978-3-319-97310-4_18
复制
发表时间：
2018-08
期刊：
影响因子：
--
通讯作者：
Tomonari Masada;A. Takasu
中科院分区：
文献类型：
--
作者：
Tomonari Masada;A. Takasu
本文提出了一个时间感知主题模型及其小批量变分推理，用于探索文档内容的时间趋势。我们的贡献是双重的。首先，为了以时间感知的方式提取主题，我们的方法使用两种向量嵌入：潜在主题的嵌入和文档时间戳的嵌入。通过结合这两个嵌入并应用softmax函数，我们可以得到与每个主题的文档时间戳一样多的单词概率分布。这种模型使我们能够提取出显著的局部趋势。其次，为了提高存储效率，将变分推理实现为最小化证据下界的小批量梯度上升。这使我们能够以类似于神经网络的方式执行参数估计。我们的方法实际上是用深度学习框架实现的。评估结果表明，使用文档时间戳可以提高测试集的困惑度，并且我们的测试集困惑度与崩溃Gibbs抽样的测试集困惑度相当，而崩溃Gibbs抽样在内存使用方面的效率低于所提出的推理。
This paper proposes a time-aware topic model and its mini-batch variational inference for exploring chronological trends in document contents. Our contribution is twofold. First, to extract topics in a time-aware manner, our method uses two vector embeddings: the embedding of latent topics and that of document timestamps. By combining these two embeddings and applying the softmax function, we have as many word probability distributions as document timestamps for each topic. This modeling enables us to extract remarkable topical trends. Second, to achieve memory efficiency, the variational inference is implemented as mini-batch gradient ascent maximizing the evidence lower bound. This enables us to perform parameter estimation in the way similar to neural networks. Our method was actually implemented with deep learning framework. The evaluation results show that we could improve test set perplexity by using document timestamps and also that our test perplexity was comparable with that of collapsed Gibbs sampling, which is less efficient in memory usage than the proposed inference.