Asymptotic Consistency of α-Rényi-Approximate Posteriors
Asymptotic Consistency of α-Rényi-Approximate Posteriors
复制标题
DOI：
--
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Prateek Jaiswal;Vinayak A. Rao;Harsha Honnappa
中科院分区：
文献类型：
--
作者：
Prateek Jaiswal;Vinayak A. Rao;Harsha Honnappa
我们研究了$\alpha$-R\'enyi近似后验的渐近一致性，一类变分贝叶斯方法，近似一个棘手的贝叶斯后验与一个易于处理的家庭的成员的分布，成员选择，以最大限度地减少$\alpha$-R\' enyi分歧从真正的后验。我们的工作的独特之处在于，我们考虑了$\alpha > 1$的设置，从而导致近似值的对数似然上限，因此比传统的变分方法具有更广泛的传播，最大限度地减少了Kullback-Liebler（KL）与后验的分歧。我们的主要结果确定了充分的条件下，一致性举行，围绕存在一个“好”的序列的分布在近似家庭，拥有，除其他外，正确的速度收敛到一个极限分布。我们进一步刻画了好的序列，证明了一个序列的分布，收敛太快，不能是一个好的序列。我们还将我们的分析扩展到$\alpha$等于1的设置，对应于反向KL发散的最小值，以及具有局部潜变量的模型。我们还用一些例子说明了好序列的存在性。我们的研究结果补充了越来越多的工作集中在变分贝叶斯方法的频率属性。
We study the asymptotic consistency properties of $\alpha$-R\'enyi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the $\alpha$-R\'enyi divergence from the true posterior. Unique to our work is that we consider settings with $\alpha > 1$, resulting in approximations that upperbound the log-likelihood, and consequently have wider spread than traditional variational approaches that minimize the Kullback-Liebler (KL) divergence from the posterior. Our primary result identifies sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family that possesses, among other properties, the right rate of convergence to a limit distribution. We further characterize the good sequence by demonstrating that a sequence of distributions that converges too quickly cannot be a good sequence. We also extend our analysis to the setting where $\alpha$ equals one, corresponding to the minimizer of the reverse KL divergence, and to models with local latent variables. We also illustrate the existence of good sequence with a number of examples. Our results complement a growing body of work focused on the frequentist properties of variational Bayesian methods.