Excess Risk Bounds for the Bayes Risk using Variational Inference in Latent Gaussian Models
Excess Risk Bounds for the Bayes Risk using Variational Inference in Latent Gaussian Models
复制标题
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Rishit Sheth;R. Khardon
中科院分区：
文献类型：
--
作者：
Rishit Sheth;R. Khardon
贝叶斯模型是机器学习中解决复杂问题的主要成功范例之一。为了处理难以处理的推理，这一领域的研究已经开发出快速有效的新的近似方法。然而，对这类近似的性能的理论分析并不是很完善。通过对一大类具有高斯潜变量的潜变量模型的变分推理算法和相关的正则化损失最小化算法的超额风险界进行了进一步的分析。我们通过证明变分算法与任何点估计预测器是竞争的，从而加强了以前的结果。与以前的工作不同，我们还给出了相同近似后验概率下{贝叶斯预报器的风险的界，而不仅仅是Gibbs预报器的风险。这些界被应用于复杂模型，包括稀疏高斯过程和相关主题模型。理论结果的补充是通过确定与贝叶斯目标的新近似，试图将风险直接降至最低。一项经验评估比较了变分算法和新算法，进一步揭示了它们的性能。
Bayesian models are established as one of the main successful paradigms for complex problems in machine learning. To handle intractable inference, research in this area has developed new approximation methods that are fast and effective. However, theoretical analysis of the performance of such approximations is not well developed. The paper furthers such analysis by providing bounds on the excess risk of variational inference algorithms and related regularized loss minimization algorithms for a large class of latent variable models with Gaussian latent variables. We strengthen previous results for variational algorithms by showing they are competitive with any point-estimate predictor. Unlike previous work, we also provide bounds on the risk of the \emph{Bayesian} predictor and not just the risk of the Gibbs predictor for the same approximate posterior. The bounds are applied in complex models including sparse Gaussian processes and correlated topic models. Theoretical results are complemented by identifying novel approximations to the Bayesian objective that attempt to minimize the risk directly. An empirical evaluation compares the variational and new algorithms shedding further light on their performance.