On Markov chain Monte Carlo methods for tall data
On Markov chain Monte Carlo methods for tall data
复制标题
DOI：
--
复制
发表时间：
2015-05
期刊：
影响因子：
--
通讯作者：
R. Bardenet;A. Doucet;C. Holmes
中科院分区：
文献类型：
--
作者：
R. Bardenet;A. Doucet;C. Holmes
马尔可夫链蒙特卡罗方法通常被认为计算过于密集，不适用于大数据应用，特别是对于包含大量$n$个单独数据点的数据集的推断，也称为高层数据集。在假设数据独立的情况下，最近在机器学习和计算统计中提出了在贝叶斯推理环境中扩大Metropolis-Hastings算法的各种方法。这些方法可以分为两类：分而治之的方法和基于次抽样的算法。本文的研究目的如下。首先，我们全面回顾了现有的文献，评论了每种方法的基本假设和理论保证。其次，通过利用我们对这些限制的理解，我们提出了一种原始的基于次抽样的方法，该方法从被证明接近感兴趣的后验分布的分布中采样，但在有利的情况下，对于某些统计模型，在每次迭代中需要不到$O(N)$数据点的似然评估。最后，到目前为止，我们只能提出基于次抽样的方法，这些方法在目标后验分布的Bernstein-von Mise近似很好的情况下表现出良好的性能。在伯恩斯坦-冯·米塞斯近似很差的情况下，开发这样的方法仍然是一个悬而未决的挑战。
Markov chain Monte Carlo methods are often deemed too computationally intensive to be of any practical use for big data applications, and in particular for inference on datasets containing a large number $n$ of individual data points, also known as tall datasets. In scenarios where data are assumed independent, various approaches to scale up the Metropolis-Hastings algorithm in a Bayesian inference context have been recently proposed in machine learning and computational statistics. These approaches can be grouped into two categories: divide-and-conquer approaches and, subsampling-based algorithms. The aims of this article are as follows. First, we present a comprehensive review of the existing literature, commenting on the underlying assumptions and theoretical guarantees of each method. Second, by leveraging our understanding of these limitations, we propose an original subsampling-based approach which samples from a distribution provably close to the posterior distribution of interest, yet can require less than $O(n)$ data point likelihood evaluations at each iteration for certain statistical models in favourable scenarios. Finally, we have only been able so far to propose subsampling-based methods which display good performance in scenarios where the Bernstein-von Mises approximation of the target posterior distribution is excellent. It remains an open challenge to develop such methods in scenarios where the Bernstein-von Mises approximation is poor.