Stochastic Variance-Reduced Hamilton Monte Carlo Methods
Stochastic Variance-Reduced Hamilton Monte Carlo Methods
复制标题
DOI：
--
复制
发表时间：
2018-02
期刊：
影响因子：
--
通讯作者：
Difan Zou;Pan Xu;Quanquan Gu
中科院分区：
文献类型：
--
作者：
Difan Zou;Pan Xu;Quanquan Gu
提出了一种快速随机汉密尔顿蒙特卡罗（HMC）方法，用于光滑强对数凹分布的抽样。我们所提出的方法的核心是一个方差减少技术的灵感来自随机优化的最新进展。我们证明，为了在2-Wasserstein距离中达到$\n $的精度，我们的算法达到了$\tilde O\big（n+\kappa^{2}d^{1/2}/\big +\kappa^{4/3}d^{1/3}n^{2/3}/\big）$梯度复杂度（即，分量梯度评估的数量），其在宽范围内优于最新的HMC和随机梯度HMC方法。我们还扩展了我们的算法，从光滑和一般的对数凹分布的采样，并证明了相应的梯度复杂性。在合成数据和真实的数据上的实验证明了该算法的上级性能。
We propose a fast stochastic Hamilton Monte Carlo (HMC) method, for sampling from a smooth and strongly log-concave distribution. At the core of our proposed method is a variance reduction technique inspired by the recent advance in stochastic optimization. We show that, to achieve $\epsilon$ accuracy in 2-Wasserstein distance, our algorithm achieves $\tilde O\big(n+\kappa^{2}d^{1/2}/\epsilon+\kappa^{4/3}d^{1/3}n^{2/3}/\epsilon^{2/3}\big)$ gradient complexity (i.e., number of component gradient evaluations), which outperforms the state-of-the-art HMC and stochastic gradient HMC methods in a wide regime. We also extend our algorithm for sampling from smooth and general log-concave distributions, and prove the corresponding gradient complexity as well. Experiments on both synthetic and real data demonstrate the superior performance of our algorithm.