CONCENTRATION OF TEMPERED POSTERIORS AND OF THEIR VARIATIONAL APPROXIMATIONS
CONCENTRATION OF TEMPERED POSTERIORS AND OF THEIR VARIATIONAL APPROXIMATIONS
复制标题
DOI：
10.1214/19-aos1855
复制
发表时间：
2020-06-01
影响因子：
4.5
通讯作者：
Ridgway, James
中科院分区：
文献类型：
--
作者：
Alquier, Pierre;Ridgway, James
虽然贝叶斯方法在统计学和机器学习中非常流行，但将其应用于海量数据集通常具有挑战性。经典的MCMC算法在模型维数和样本量都很大时速度非常慢。变分贝叶斯方法的目的是近似后验分布在一个易于处理的家庭F。因此，MCMC被更快数量级的优化算法所取代。VB方法已经应用于计算要求高的应用中，例如协同过滤，图像和视频处理或NLP等。然而，尽管在实践中取得了很好的结果，这些近似的理论性质是未知的。我们提出了一个一般的预言不等式，它将VB逼近的质量与先验π和F的结构联系起来。我们提供了一个简单的条件，允许从这个预言不等式的收敛速度。我们将我们的理论应用于各种例子。首先，我们证明了对数Lipschitz似然参数模型，高斯VB导致有效的算法和一致的估计。然后，我们研究一个高维的例子：矩阵完成，和一个非参数的例子：密度估计。
While Bayesian methods are extremely popular in statistics and machine learning, their application to massive data sets is often challenging, when possible at all. The classical MCMC algorithms are prohibitively slow when both the model dimension and the sample size are large. Variational Bayesian methods aim at approximating the posterior by a distribution in a tractable family F. Thus, MCMC are replaced by an optimization algorithm which is orders of magnitude faster. VB methods have been applied in such computationally demanding applications as collaborative filtering, image and video processing or NLP to name a few. However, despite nice results in practice, the theoretical properties of these approximations are not known. We propose a general oracle inequality that relates the quality of the VB approximation to the prior pi and to the structure of F. We provide a simple condition that allows to derive rates of convergence from this oracle inequality. We apply our theory to various examples. First, we show that for parametric models with log-Lipschitz likelihood, Gaussian VB leads to efficient algorithms and consistent estimators. We then study a high-dimensional example: matrix completion, and a nonparametric example: density estimation.