parallelMCMCcombine: an R package for bayesian methods for big data and analytics.
parallelMCMCcombine: an R package for bayesian methods for big data and analytics.
复制标题
DOI：
10.1371/journal.pone.0108425
复制
发表时间：
2014
期刊：
影响因子：
3.7
通讯作者：
Conlon EM
中科院分区：
文献类型：
--
作者：
Miroshnikov A;Conlon EM
大数据和分析研究的最新进展提供了大量的大型数据集，由于计算机内存或存储大小的限制，这些数据集太大而无法进行整体分析。新的贝叶斯方法已经开发的数据集是大的，只是由于大样本量。这些方法将大数据集划分为子集，并对子集执行独立的贝叶斯马尔可夫链蒙特卡罗分析。该方法然后结合联合收割机的独立子集后验样本估计后验密度给定的完整的数据集。这些方法被证明是有效的贝叶斯模型，包括逻辑回归模型，高斯混合模型和分层模型。在这里，我们介绍了R包parallelMCMCcombine，它执行了四种用于组合独立子集后验样本的技术。我们说明了每一种方法使用贝叶斯逻辑回归模型的模拟数据和贝叶斯伽马模型的真实的数据，我们还展示了功能和功能的R包。该软件包假设用户已执行贝叶斯分析，并已在软件包外生成独立的次后验样本。该方法主要适用于连续参数空间中具有固定维数未知参数的模型。我们设想这个工具将使研究人员能够探索各种方法，以满足其特定的应用，并将有助于在这个快速发展的领域的未来进展。
Recent advances in big data and analytics research have provided a wealth of large data sets that are too big to be analyzed in their entirety, due to restrictions on computer memory or storage size. New Bayesian methods have been developed for data sets that are large only due to large sample sizes. These methods partition big data sets into subsets and perform independent Bayesian Markov chain Monte Carlo analyses on the subsets. The methods then combine the independent subset posterior samples to estimate a posterior density given the full data set. These approaches were shown to be effective for Bayesian models including logistic regression models, Gaussian mixture models and hierarchical models. Here, we introduce the R package parallelMCMCcombine which carries out four of these techniques for combining independent subset posterior samples. We illustrate each of the methods using a Bayesian logistic regression model for simulation data and a Bayesian Gamma model for real data; we also demonstrate features and capabilities of the R package. The package assumes the user has carried out the Bayesian analysis and has produced the independent subposterior samples outside of the package. The methods are primarily suited to models with unknown parameters of fixed dimension that exist in continuous parameter spaces. We envision this tool will allow researchers to explore the various methods for their specific applications and will assist future progress in this rapidly developing field.
登录
查看更多内容
影响因子：
2.5
作者：
Smola, Alexander;Narayanamurthy, Shravan
通讯作者：
Narayanamurthy, Shravan
影响因子：
7.5
作者：
Laskey, KB;Myers, JW
通讯作者：
Myers, JW
影响因子：
4.5
作者：
HJORT, NL;GLAD, IK
通讯作者：
GLAD, IK
影响因子：
1.2
作者：
Duong, T;Hazelton, ML
通讯作者：
Hazelton, ML
影响因子：
2.2
作者：
Lunn, DJ;Thomas, A;Spiegelhalter, D
通讯作者：
Spiegelhalter, D