Conjugate-Computation Variational Inference: Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
Conjugate-Computation Variational Inference: Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
复制标题
共轭计算变分推理：将非共轭模型中的变分推理转换为共轭模型中的推理
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Wu Lin
中科院分区：
文献类型：
--
作者：
M. E. Khan;Wu Lin
在既包含共轭项又包含非共轭项的模型中，变分推理在计算上具有挑战性。专门为共轭模型设计的方法，尽管计算效率很高，但发现很难处理非共轭项。另一方面，随机梯度法可以处理非共轭项，但通常忽略了模型的共轭结构，这可能会导致收敛速度较慢。在本文中，我们提出了一种新的算法，称为共轭计算变分推理(CVI)，它结合了这两个领域的优点--它对共轭项使用共轭计算，对其余项使用随机梯度。我们在均值参数空间使用随机镜像下降方法，然后将每个梯度步长表示为共轭模型中的变分推理，从而推导出该算法。我们证明了我们的算法对一大类模型的适用性，并证明了它的收敛。实验结果表明，我们的方法比忽略模型共轭结构的方法收敛速度快得多。
Variational inference is computationally challenging in models that contain both conjugate and non-conjugate terms. Methods specifically designed for conjugate models, even though computationally efficient, find it difficult to deal with non-conjugate terms. On the other hand, stochastic-gradient methods can handle the non-conjugate terms but they usually ignore the conjugate structure of the model which might result in slow convergence. In this paper, we propose a new algorithm called Conjugate-computation Variational Inference (CVI) which brings the best of the two worlds together -- it uses conjugate computations for the conjugate terms and employs stochastic gradients for the rest. We derive this algorithm by using a stochastic mirror-descent method in the mean-parameter space, and then expressing each gradient step as a variational inference in a conjugate model. We demonstrate our algorithm's applicability to a large class of models and establish its convergence. Our experimental results show that our method converges much faster than the methods that ignore the conjugate structure of the model.