An algorithm for distributed Bayesian inference
An algorithm for distributed Bayesian inference
复制标题
DOI：
10.1002/sta4.432
复制
发表时间：
2022-12-01
期刊：
影响因子：
1.7
通讯作者：
Srivastava, Sanvesh
中科院分区：
文献类型：
--
作者：
Shyamalkumar, Nariankadu D.;Srivastava, Sanvesh
蒙特卡罗算法，如马尔可夫链蒙特卡罗（MCMC）和汉密尔顿蒙特卡罗（HMC），通常用于贝叶斯推理;然而，这些算法在大规模数据设置中非常慢，因为它们需要在每次迭代中多次通过完整数据。为了解决这个问题，我们开发了一个可扩展的扩展这些算法，使用分治（D&C）技术，将数据划分为一个足够大的数量的子集，并行绘制参数的子集使用功率的可能性，并产生Monte Carlo绘制的参数相结合，从每个子集获得的参数绘制。组合参数绘制起到了原始采样算法绘制的作用。我们的主要贡献有两方面。首先，我们通过不同的模拟和真实的数据分析集中在广义线性模型（GLM），我们的分布式算法提供了可比的结果，目前国家的最先进的D&C算法的统计精度和计算效率。其次，为我们的经验观察提供理论支持，我们确定的规则性假设下，所提出的算法导致渐近最优推理。我们还提供了说明性的例子，重点是正常的线性和逻辑回归，我们的D&C算法的部分是分析听话。
Monte Carlo algorithms, such as Markov chain Monte Carlo (MCMC) and Hamiltonian Monte Carlo (HMC), are routinely used for Bayesian inference; however, these algorithms are prohibitively slow in massive data settings because they require multiple passes through the full data in every iteration. Addressing this problem, we develop a scalable extension of these algorithms using the divide-and-conquer (D&C) technique that divides the data into a sufficiently large number of subsets, draws parameters in parallel on the subsets using a powered likelihood and produces Monte Carlo draws of the parameter by combining parameter draws obtained from each subset. The combined parameter draws play the role of draws from the original sampling algorithm. Our main contributions are twofold. First, we demonstrate through diverse simulated and real data analyses focusing on generalized linear models (GLMs) that our distributed algorithm delivers comparable results as the current state-of-the-art D&C algorithms in terms of statistical accuracy and computational efficiency. Second, providing theoretical support for our empirical observations, we identify regularity assumptions under which the proposed algorithm leads to asymptotically optimal inference. We also provide illustrative examples focusing on normal linear and logistic regressions where parts of our D&C algorithm are analytically tractable.