Efficient Collapsed Gibbs Sampling for Latent Dirichlet Allocation
Efficient Collapsed Gibbs Sampling for Latent Dirichlet Allocation
复制标题
DOI：
--
复制
发表时间：
2010-10
期刊：
影响因子：
--
通讯作者：
Han Xiao;T. Stibor
中科院分区：
文献类型：
--
作者：
Han Xiao;T. Stibor
在概率生成模型（如潜在狄利克雷分配）中，压缩吉布斯采样是一种常用的近似难处理积分的方法。然而，这种采样方法的关键缺点是计算复杂度高，这使得它在大数据集上的应用受到限制。我们提出了一种新的动态采样策略，以显着提高效率的折叠吉布斯采样。从效率、收敛性和困惑性三个方面对该策略进行了探讨。此外，我们提出了一个直接的并行化，以进一步提高效率。最后，我们支持我们提出的改进与不同规模的数据集的比较研究。
Collapsed Gibbs sampling is a frequently applied method to approximate intractable integrals in probabilistic generative models such as latent Dirichlet allocation. This sampling method has however the crucial drawback of high computational complexity, which makes it limited applicable on large data sets. We propose a novel dynamic sampling strategy to significantly improve the efficiency of collapsed Gibbs sampling. The strategy is explored in terms of efficiency, convergence and perplexity. Besides, we present a straight-forward parallelization to further improve the efficiency. Finally, we underpin our proposed improvements with a comparative study on different scale data sets.