Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling
Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling
复制标题
DOI：
10.1214/ss/1028905934
复制
发表时间：
1998-05
影响因子：
5.7
通讯作者：
A. Gelman;X. Meng
中科院分区：
文献类型：
--
作者：
A. Gelman;X. Meng
计算概率模型的归一化常数（比）是许多统计和科学研究的基本计算问题。蒙特卡罗模拟是一种有效的技术，特别是对于复杂的高维模型。本文旨在通过建立理论联系和说明它们在统计问题中的应用，使一般统计读者注意到一些源于理论物理的有效方法，同时从更统计的角度来解释这些方法。我们表明，接受率方法和热力学积分的重要性抽样，这是最熟悉的统计观众的自然概括。前者通过使用单个“桥”密度来概括重要性抽样，因此是孟和王意义上的桥抽样的情况。热力学积分，在数值分析文献中也被称为绪方的高维积分方法，对应于使用无限多个连续连接的桥（因此是一条“路径”）。我们的路径采样公式为热力学积分提供了更大的灵活性和潜在的效率，并且最佳路径的搜索与Jeffreys先验密度以及两个密度之间的Rao和Hellinger距离有密切的联系。我们提供了一个信息丰富的理论例子，以及两个经验的例子（涉及17至70维集成），以说明路径采样的潜力和实现。我们还讨论了一些开放的问题。
Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, es- pecially with complex and high-dimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to ex- plore these methods from a more statistical perspective, through estab- lishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single "bridge" density and is thus a case of bridge sampling in the sense of Meng and Wong. Thermodynamic integration, which is also known in the numerical analysis literature as Ogata's method for high-dimensional integration, corresponds to the use of infinitely many and continuously connected bridges (and thus a "path"). Our path sampling formulation offers more flexibility and thus potential efficiency to thermodynamic integration, and the search of op- timal paths turns out to have close connections with the Jeffreys prior density and the Rao and Hellinger distances between two densities. We provide an informative theoretical example as well as two empirical ex- amples (involving 17- to 70-dimensional integrations) to illustrate the potential and implementation of path sampling. We also discuss some open problems.