Variational Hamiltonian Monte Carlo via Score Matching.
Variational Hamiltonian Monte Carlo via Score Matching.
复制标题
DOI：
10.1214/17-ba1060
复制
发表时间：
2016-02
影响因子：
4.4
通讯作者：
Cheng Zhang;B. Shahbaba;Hongkai Zhao
中科院分区：
文献类型：
--
作者：
Cheng Zhang;B. Shahbaba;Hongkai Zhao
传统上，计算贝叶斯统计领域被分为两个主要的子领域：变分方法和马尔可夫链蒙特卡罗(MCMC)。然而，近年来，为了提高变分贝叶斯推理和MCMC模拟的整体精度和计算效率，人们提出了几种基于变分贝叶斯推理和MCMC模拟的方法。这种快速评估和灵活近似的结合为设计可扩展的贝叶斯推理方法提供了一种很有前途的手段。在本文中，我们探索了将变分近似与最先进的MCMC方法--哈密顿蒙特卡罗(HMC)相结合的可能性，以减少采样过程中所需的昂贵计算，这是HMC在大数据问题中许多应用的瓶颈。为此，我们利用参数空间的正则性，利用适当随机基的优化加性模型，通过一个快速而灵活的代理函数来构造目标分布的自由形式近似，该模型也可以看作是一个单隐层前馈神经网络。代理函数提供了足够精确的近似，同时允许采样过程中的快速计算，从而产生了高效的近似贝叶斯推理算法。我们使用合成数据问题和真实数据问题来演示我们所提出的方法的优点。
Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their overall accuracy and computational efficiency. This marriage of fast evaluation and flexible approximation provides a promising means of designing scalable Bayesian inference methods. In this paper, we explore the possibility of incorporating variational approximation into a state-of-the-art MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required expensive computation involved in the sampling procedure, which is the bottleneck for many applications of HMC in big data problems. To this end, we exploit the regularity in parameter space to construct a free-form approximation of the target distribution by a fast and flexible surrogate function using an optimized additive model of proper random basis, which can also be viewed as a single-hidden layer feedforward neural network. The surrogate function provides sufficiently accurate approximation while allowing for fast computation in the sampling procedure, resulting in an efficient approximate Bayesian inference algorithm. We demonstrate the advantages of our proposed method using both synthetic and real data problems.