Collapsed Variational Inference for HDP
Collapsed Variational Inference for HDP
复制标题
DOI：
--
复制
发表时间：
2007-12
期刊：
影响因子：
--
通讯作者：
Y. Teh;Kenichi Kurihara;M. Welling
中科院分区：
文献类型：
--
作者：
Y. Teh;Kenichi Kurihara;M. Welling
各种各样的狄利克雷多项式的“主题”模型，近年来发现有趣的应用。虽然吉布斯抽样仍然是一个重要的推理方法，在这样的模型，变分技术有一定的优势，如容易评估的收敛性，易于优化，而不需要保持详细的平衡，一个边界的边际可能性，并与主题可识别性问题的侧步。迄今为止最精确的变分技术，即塌陷变分潜在狄利克雷分配，不处理模型选择，也不包括超参数的推断。我们解决这两个问题，通过推广的技术，获得第一个变分算法来处理层次Dirichlet过程和处理超参数的Dirichlet变量。实验结果表明，在准确性上有显著提高。
A wide variety of Dirichlet-multinomial 'topic' models have found interesting applications in recent years. While Gibbs sampling remains an important method of inference in such models, variational techniques have certain advantages such as easy assessment of convergence, easy optimization without the need to maintain detailed balance, a bound on the marginal likelihood, and side-stepping of issues with topic-identifiability. The most accurate variational technique thus far, namely collapsed variational latent Dirichlet allocation, did not deal with model selection nor did it include inference for hyperparameters. We address both issues by generalizing the technique, obtaining the first variational algorithm to deal with the hierarchical Dirichlet process and to deal with hyperparameters of Dirichlet variables. Experiments show a significant improvement in accuracy.