Provable Smoothness Guarantees for Black-Box Variational Inference
Provable Smoothness Guarantees for Black-Box Variational Inference
复制标题
黑盒变分推理的可证明平滑性保证
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Justin Domke
中科院分区：
文献类型：
--
作者：
Justin Domke
黑盒变分推理试图通过对简单分布的参数进行基于梯度的优化来近似复杂的目标分布。可证明的收敛保证需要目标的结构属性。本文证明了对于位置-尺度族近似，如果目标是M-Lipschitz光滑的，那么如果熵被排除，目标也是M-Lipschitz光滑的。关键的证明思想是在一定的内积空间中描述梯度，从而允许使用贝塞尔不等式。这一结果提供了深入了解如何参数化分布，给出了最优参数的位置范围，并且是收敛保证的关键成分。
Black-box variational inference tries to approximate a complex target distribution though a gradient-based optimization of the parameters of a simpler distribution. Provable convergence guarantees require structural properties of the objective. This paper shows that for location-scale family approximations, if the target is M-Lipschitz smooth, then so is the objective, if the entropy is excluded. The key proof idea is to describe gradients in a certain inner-product space, thus permitting use of Bessel's inequality. This result gives insight into how to parameterize distributions, gives bounds the location of the optimal parameters, and is a key ingredient for convergence guarantees.