On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods
On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods
复制标题
DOI：
10.1214/15-sts523
复制
发表时间：
2015-11-01
影响因子：
5.7
通讯作者：
Simpson, Daniel
中科院分区：
文献类型：
--
作者：
Lyne, Anne-Marie;Girolami, Mark;Simpson, Daniel
大量的统计模型是“双重难以处理的”：作为模型参数函数的似然归一化项和边际似然（模型证据）都是难以处理的。这意味着从后验抽样的标准推理技术，如马尔可夫链蒙特卡罗（MCMC），不能使用。例子包括，但不限于，大量高斯马尔可夫随机场，自定义模型和指数随机图模型。一些基于MCMC技术的近似方案，近似贝叶斯计算（ABC）或后验的解析近似已经被提出，这些在这里进行了回顾。精确的MCMC格式，可以应用于双难处理分布的一个子集，也已经发展，并在本文中进行了描述。到目前为止，还没有一种通用的方法可以适用于所有具有双难处理后验的模型。此外，从物理文献中获得灵感，我们研究了一种基于将难处理似然表示为无穷级数的替代方法。然后，通过俄罗斯轮盘赌抽样对序列进行有限时间随机截断，可以获得无偏似然估计，尽管估计不一定是正的。利用量子色动力学文献的结果，允许在伪边际MCMC方案中使用可能的负估计，从而保留了对后验分布的期望。该方法回顾了著名的例子，如伊辛模型中的参数，d球上Fisher Bingham分布的后验和描述臭氧柱数据的大规模高斯马尔可夫随机场模型。这导致了对方法学优缺点的批判性评估，并指出了正在进行的研究。
A large number of statistical models are "doubly-intractable": the likelihood normalising term, which is a function of the model parameters, is intractable, as well as the marginal likelihood (model evidence). This means that standard inference techniques to sample from the posterior, such as Markov chain Monte Carlo (MCMC), cannot be used. Examples include, but are not confined to, massive Gaussian Markov random fields, autologistic models and Exponential random graph models. A number of approximate schemes based on MCMC techniques, Approximate Bayesian computation (ABC) or analytic approximations to the posterior have been suggested, and these are reviewed here. Exact MCMC schemes, which can be applied to a subset of doubly-intractable distributions, have also been developed and are described in this paper. As yet, no general method exists which can be applied to all classes of models with doubly-intractable posteriors. In addition, taking inspiration from the Physics literature, we study an alternative method based on representing the intractable likelihood as an infinite series. Unbiased estimates of the likelihood can then be obtained by finite time stochastic truncation of the series via Russian Roulette sampling, although the estimates are not necessarily positive. Results from the Quantum Chromodynamics literature are exploited to allow the use of possibly negative estimates in a pseudo-marginal MCMC scheme such that expectations with respect to the posterior distribution are preserved. The methodology is reviewed on well-known examples such as the parameters in Ising models, the posterior for Fisher Bingham distributions on the d-Sphere and a large-scale Gaussian Markov Random Field model describing the Ozone Column data. This leads to a critical assessment of the strengths and weaknesses of the methodology with pointers to ongoing research.