Variational Bayes under Model Misspecification
Variational Bayes under Model Misspecification
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Yixin Wang;D. Blei
中科院分区：
文献类型：
--
作者：
Yixin Wang;D. Blei
变分贝叶斯（VB）是一种可扩展的替代马尔可夫链蒙特卡罗（MCMC）的贝叶斯后验推理。虽然流行，VB附带的理论保证很少，其中大部分集中在良好指定的模型。然而，模型在实践中很少被很好地指定。在这项工作中，我们研究了VB下的模型误指定。我们证明了VB后验是渐近正态的，并以最小化Kullback-Leibler（KL）发散到真实数据生成分布的值为中心。此外，VB后验均值集中在相同的值，也是渐近正态的。这些结果将变分伯恩斯坦-冯米塞斯定理[29]推广到错误指定的模型。作为这些结果的结果，我们发现，在VB后验预测分布的模型误设定误差占主导地位的变分逼近误差。它解释了广泛观察到的现象，即VB实现了与MCMC相当的预测精度，即使VB使用近似族。作为例证，我们研究VB下的三种形式的模型误指定，从模型过度/欠分散潜在的维度误指定。我们进行了两个模拟研究，证明了理论结果。
Variational Bayes (VB) is a scalable alternative to Markov chain Monte Carlo (MCMC) for Bayesian posterior inference. Though popular, VB comes with few theoretical guarantees, most of which focus on well-specified models. However, models are rarely well-specified in practice. In this work, we study VB under model misspecification. We prove the VB posterior is asymptotically normal and centers at the value that minimizes the Kullback-Leibler (KL) divergence to the true data-generating distribution. Moreover, the VB posterior mean centers at the same value and is also asymptotically normal. These results generalize the variational Bernstein--von Mises theorem [29] to misspecified models. As a consequence of these results, we find that the model misspecification error dominates the variational approximation error in VB posterior predictive distributions. It explains the widely observed phenomenon that VB achieves comparable predictive accuracy with MCMC even though VB uses an approximating family. As illustrations, we study VB under three forms of model misspecification, ranging from model over-/under-dispersion to latent dimensionality misspecification. We conduct two simulation studies that demonstrate the theoretical results.