On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations
On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations
复制标题
DOI：
--
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Marco Avella Medina;J. M. Olea;Cynthia Rush;Amilcar Velez
中科院分区：
文献类型：
--
作者：
Marco Avella Medina;J. M. Olea;Cynthia Rush;Amilcar Velez
$\alpha$-后验及其变分近似通过降低可能性和引入变分近似误差来扭曲标准后验推理。我们表明，如果适当调整这种扭曲，当存在潜在的参数模型规格错误时，可以减少与真实但可能不可行的后验分布的Kullback-Leibler （KL）散度。为了证明这一点，我们推导了一个Bernstein-von Mises定理，该定理表明$\alpha$-后验的总变异距离和它们对极限高斯分布的变分近似是收敛的。我们使用这些分布来评估真实后验和报告后验之间的KL差异。我们通过选择严格小于1的$\alpha$来最小化这种散度，假设模型错误规范的概率非常小。当规格偏差越严重时，优化值越小。优化后的KL散度在错配程度上呈对数增长，而不是像通常的后验那样呈线性增长。
$\alpha$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce the Kullback-Leibler (KL) divergence from the true, but perhaps infeasible, posterior distribution when there is potential parametric model misspecification. To make this point, we derive a Bernstein-von Mises theorem showing convergence in total variation distance of $\alpha$-posteriors and their variational approximations to limiting Gaussian distributions. We use these distributions to evaluate the KL divergence between true and reported posteriors. We show this divergence is minimized by choosing $\alpha$ strictly smaller than one, assuming there is a vanishingly small probability of model misspecification. The optimized value becomes smaller as the the misspecification becomes more severe. The optimized KL divergence increases logarithmically in the degree of misspecification and not linearly as with the usual posterior.