α-VARIATIONAL INFERENCE WITH STATISTICAL GUARANTEES
α-VARIATIONAL INFERENCE WITH STATISTICAL GUARANTEES
复制标题
DOI：
10.1214/19-aos1827
复制
发表时间：
2020-04-01
影响因子：
4.5
通讯作者：
Bhattacharya, Anirban
中科院分区：
文献类型：
--
作者：
Yang, Yun;Pati, Debdeep;Bhattacharya, Anirban
我们为贝叶斯后验分布的一系列变分近似（称为 alpha-VB）提供统计保证，它与文献中调和后验的变分近似密切相关。标准变分近似是 alpha = 1 的 alpha-VB 的特例。当 alpha 是 (0, 1] 的元素时，开发了一类新颖的变分不等式，用于将变分近似下的贝叶斯风险与变分优化问题中的目标函数联系起来，这意味着最大化变分推理中的证据下界具有最小化变分密度族内的贝叶斯风险的效果。不等式意味着从 alpha-VB 过程构建的点估计在各种问题中以最佳速率收敛到真实参数，我们用许多例子说明了我们的一般理论，包括带有尖峰和平板先验的（低）高维贝叶斯线性回归的平均场变分逼近、高斯混合模型和潜在狄利克雷分配。
We provide statistical guarantees for a family of variational approximations to Bayesian posterior distributions, called alpha-VB, which has close connections with variational approximations of tempered posteriors in the literature. The standard variational approximation is a special case of alpha-VB with alpha = 1. When alpha is an element of (0, 1], a novel class of variational inequalities are developed for linking the Bayes risk under the variational approximation to the objective function in the variational optimization problem, implying that maximizing the evidence lower bound in variational inference has the effect of minimizing the Bayes risk within the variational density family. Operating in a frequentist setup, the variational inequalities imply that point estimates constructed from the alpha-VB procedure converge at an optimal rate to the true parameter in a wide range of problems. We illustrate our general theory with a number of examples, including the mean-field variational approximation to (low)-high-dimensional Bayesian linear regression with spike and slab priors, Gaussian mixture models and latent Dirichlet allocation.