Spherical Hamiltonian Monte Carlo for Constrained Target Distributions
Spherical Hamiltonian Monte Carlo for Constrained Target Distributions
复制标题
DOI：
--
复制
发表时间：
2013-09
期刊：
影响因子：
--
通讯作者：
Shiwei Lan;Bo Zhou;B. Shahbaba
中科院分区：
文献类型：
--
作者：
Shiwei Lan;Bo Zhou;B. Shahbaba
在机器学习中，具有约束概率分布的统计模型非常丰富。一些例子包括具有范数约束的回归模型(例如，套索)、概率模型、许多Copula模型和潜在狄利克雷分配(LDA)模型。对于常用的抽样算法来说，涉及受限区域内的概率分布的贝叶斯推理可能是相当具有挑战性的。针对这类问题，我们提出了一种新的马尔可夫链蒙特卡罗(MCMC)方法，它为处理边界条件提供了一个通用的、计算高效的框架。我们的方法首先将参数的D维约束区域映射到单位球[公式：参见文本]，然后将其扩充到D维球SD，使得原始边界对应于SD的赤道。这样，我们的方法通过在球体上自由移动来隐式地处理约束，从而生成在映射回原始空间时保持在边界内的建议。为了提高算法的计算效率，我们将动力学分成几个部分，使得得到的分裂动力学具有球面上测地线流的部分解析解。我们将我们的方法应用于几个例子，包括截断高斯、贝叶斯套索、贝叶斯桥回归和用于识别多个神经元之间的同步性的Copula模型。我们的结果表明，该方法可以提供一个自然而有效的框架来处理几种类型的目标分布约束。
Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit models, many copula models, and Latent Dirichlet Allocation (LDA) models. Bayesian inference involving probability distributions confined to constrained domains could be quite challenging for commonly used sampling algorithms. For such problems, we propose a novel Markov Chain Monte Carlo (MCMC) method that provides a general and computationally efficient framework for handling boundary conditions. Our method first maps the D-dimensional constrained domain of parameters to the unit ball [Formula: see text], then augments it to a D-dimensional sphere SD such that the original boundary corresponds to the equator of SD . This way, our method handles the constraints implicitly by moving freely on the sphere generating proposals that remain within boundaries when mapped back to the original space. To improve the computational efficiency of our algorithm, we divide the dynamics into several parts such that the resulting split dynamics has a partial analytical solution as a geodesic flow on the sphere. We apply our method to several examples including truncated Gaussian, Bayesian Lasso, Bayesian bridge regression, and a copula model for identifying synchrony among multiple neurons. Our results show that the proposed method can provide a natural and efficient framework for handling several types of constraints on target distributions.