Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets.
Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets.
复制标题
大型数据集一次性线性回归中的贝叶斯变量选择。
DOI：
10.1145/2629617
复制
发表时间：
2014
影响因子：
3.6
通讯作者：
Baladandayuthapani,Veerabhadran
中科院分区：
文献类型：
--
作者：
Ordonez,Carlos;Garcia-Alvarado,Carlos;Baladandayuthapani,Veerabhadran
贝叶斯模型一般采用马尔可夫链蒙特卡罗（MCMC）方法计算。MCMC方法的主要缺点是需要大量的迭代来采样模型参数的后验分布，特别是对于大型数据集。另一方面，由于其组合搜索空间，变量选择仍然是一个具有挑战性的问题，其中贝叶斯模型是一个有前途的解决方案。在本工作中，我们研究了如何加速贝叶斯模型在线性回归中变量选择的计算。我们提出了一种快速Gibbs采样算法，这是一种广泛使用的MCMC方法，它包含了几种优化。我们对回归系数使用Zellner先验，对方差使用不当先验，并使用共轭先验高斯分布，这使得数据集可以一次汇总，从而利用了一组增强的充分统计数据。之后，算法在主存中迭代。用稀疏二值向量索引足够的统计量，从而有效地计算基于选定变量的矩阵投影。发现的变量子集概率，选择和丢弃每个变量，都存储在哈希表中，以便在未来的迭代中快速检索。我们研究如何将我们的算法集成到数据库管理系统（DBMS）中，利用聚合用户定义函数进行并行数据汇总和存储过程来使用数组操作矩阵。使用真实数据集进行的实验评估评估了准确性和时间性能，将基于dbms的算法与R包进行了比较。我们的算法被证明可以产生准确的结果，在数据集大小上线性扩展，并且运行速度比R包快几个数量级。
Bayesian models are generally computed with Markov Chain Monte Carlo (MCMC) methods. The main disadvantage of MCMC methods is the large number of iterations they need to sample the posterior distributions of model parameters, especially for large datasets. On the other hand, variable selection remains a challenging problem due to its combinatorial search space, where Bayesian models are a promising solution. In this work, we study how to accelerate Bayesian model computation for variable selection in linear regression. We propose a fast Gibbs sampler algorithm, a widely used MCMC method that incorporates several optimizations. We use a Zellner prior for the regression coefficients, an improper prior on variance, and a conjugate prior Gaussian distribution, which enable dataset summarization in one pass, thus exploiting an augmented set of sufficient statistics. Thereafter, the algorithm iterates in main memory. Sufficient statistics are indexed with a sparse binary vector to efficiently compute matrix projections based on selected variables. Discovered variable subsets probabilities, selecting and discarding each variable, are stored on a hash table for fast retrieval in future iterations. We study how to integrate our algorithm into a Database Management System (DBMS), exploiting aggregate User-Defined Functions for parallel data summarization and stored procedures to manipulate matrices with arrays. An experimental evaluation with real datasets evaluates accuracy and time performance, comparing our DBMS-based algorithm with the R package. Our algorithm is shown to produce accurate results, scale linearly on dataset size, and run orders of magnitude faster than the R package.