On Sparse Variational Methods and the Kullback-Leibler Divergence between Stochastic Processes
On Sparse Variational Methods and the Kullback-Leibler Divergence between Stochastic Processes
复制标题
DOI：
10.17863/cam.15597
复制
发表时间：
2015-04
期刊：
影响因子：
--
通讯作者：
A. G. Matthews;J. Hensman;Richard E. Turner;Zoubin Ghahramani
中科院分区：
文献类型：
--
作者：
A. G. Matthews;J. Hensman;Richard E. Turner;Zoubin Ghahramani
学习诱导变量的变分框架(Titsias，2009a)对高斯过程文献产生了很大影响。该框架可以被解释为使近似和后验过程之间严格定义的Kullback-Leibler发散最小化。据我们所知，到目前为止，这种联系在文献中还没有被提及。在这篇文章中，我们对这一主题的文献进行了大量的概括。我们给出了无穷指标集结果的一个新证明，它允许非数据点的诱导点和依赖于所有函数值的似然。然后，我们讨论了增广指标集，并证明了与前人的工作相反，增广的边际一致性不足以保证变分推理与原模型的一致性。然后，我们刻画了这样一个保证可以获得的额外条件。最后，我们展示了我们的框架如何揭示了关于COX过程的域间稀疏逼近和稀疏逼近。
The variational framework for learning inducing variables (Titsias, 2009a) has had a large impact on the Gaussian process literature. The framework may be interpreted as minimizing a rigorously defined Kullback-Leibler divergence between the approximating and posterior processes. To our knowledge this connection has thus far gone unremarked in the literature. In this paper we give a substantial generalization of the literature on this topic. We give a new proof of the result for infinite index sets which allows inducing points that are not data points and likelihoods that depend on all function values. We then discuss augmented index sets and show that, contrary to previous works, marginal consistency of augmentation is not enough to guarantee consistency of variational inference with the original model. We then characterize an extra condition where such a guarantee is obtainable. Finally we show how our framework sheds light on interdomain sparse approximations and sparse approximations for Cox processes.