Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation
Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation
复制标题
DOI：
--
复制
发表时间：
2006-12
期刊：
影响因子：
--
通讯作者：
Kazuho Watanabe;Sumio Watanabe
中科院分区：
文献类型：
--
作者：
Kazuho Watanabe;Sumio Watanabe
贝叶斯学习在许多数据建模问题中得到了广泛的应用并被证明是有效的。然而，其中涉及的计算需要巨大的成本，并且通常不能精确地执行。变分贝叶斯方法作为贝叶斯学习的一种近似方法，在许多应用中提供了计算上的易处理性和良好的泛化性能。变分贝叶斯学习本身的性质和能力还没有得到澄清。它仍然是未知的如何很好的近似变分贝叶斯方法可以实现。本文讨论了高斯混合模型的变分贝叶斯学习问题，给出了变分随机复杂度的上下界。变分随机复杂性，它对应于最小变分自由能和贝叶斯证据的下界，不仅在解决模型选择问题中变得重要，而且还使我们能够讨论变分贝叶斯方法作为真实贝叶斯学习的近似的准确性。
Bayesian learning has been widely used and proved to be effective in many data modeling problems. However, computations involved in it require huge costs and generally cannot be performed exactly. The variational Bayesian approach, proposed as an approximation of Bayesian learning, has provided computational tractability and good generalization performance in many applications. The properties and capabilities of variational Bayesian learning itself have not been clarified yet. It is still unknown how good approximation the variational Bayesian approach can achieve. In this paper, we discuss variational Bayesian learning of Gaussian mixture models and derive upper and lower bounds of variational stochastic complexities. The variational stochastic complexity, which corresponds to the minimum variational free energy and a lower bound of the Bayesian evidence, not only becomes important in addressing the model selection problem, but also enables us to discuss the accuracy of the variational Bayesian approach as an approximation of true Bayesian learning.