Variational Inference for Dirichlet Process Mixtures
Variational Inference for Dirichlet Process Mixtures
复制标题
DOI：
10.1214/06-ba104
复制
发表时间：
2006-01-01
影响因子：
4.4
通讯作者：
Jordan, Michael I.
中科院分区：
文献类型：
--
作者：
Blei, David M.;Jordan, Michael I.
Dirichlet过程(DP)混合模型是非参数贝叶斯统计的基石，而针对DP混合的蒙特卡洛马尔可夫链(MCMC)抽样方法的发展使得非参数贝叶斯方法应用于各种实际数据分析问题。然而，MCMC抽样可能非常慢，探索替代方案是很重要的。一类备选方案是由变分方法提供的，变分方法是一类将推理问题转化为优化问题的确定性算法(Opper和Saad 2001；Wainwright和Jordan 2003)。到目前为止，变分方法主要是在参数设置下探索的，特别是在指数族的形式体系中(Attias2000；Ghahramani和Beal 2001；Bleital.2003)。在本文中，我们提出了一种用于DP混合的变分推理算法。我们给出了将该算法与高斯DP混合的Gibbs采样算法进行比较的实验，并给出了一个应用于大规模图像分析问题的应用。
Dirichlet process (DP) mixture models are the cornerstone of non-parametric Bayesian statistics, and the development of Monte-Carlo Markov chain (MCMC) sampling methods for DP mixtures has enabled the application of non-parametric Bayesian methods to a variety of practical data analysis problems. However, MCMC sampling can be prohibitively slow,and it is important to explore alternatives.One class of alternatives is provided by variational methods, a class of deterministic algorithms that convert inference problems into optimization problems (Opper and Saad 2001; Wainwright and Jordan 2003).Thus far, variational methods have mainly been explored in the parametric setting, in particular within the formalism of the exponential family (Attias2000; Ghahramani and Beal 2001; Bleietal .2003).In this paper, we present a variational inference algorithm for DP mixtures.We present experiments that compare the algorithm to Gibbs sampling algorithms for DP mixtures of Gaussians and present an application to a large-scale image analysis problem.