Scalable Approximate MCMC Algorithms for the Horseshoe Prior
Scalable Approximate MCMC Algorithms for the Horseshoe Prior
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
J. Johndrow;Paulo Orenstein;A. Bhattacharya
中科院分区：
文献类型：
--
作者：
J. Johndrow;Paulo Orenstein;A. Bhattacharya
马蹄先验经常用于高维模型的贝叶斯分析，并已被证明在真实稀疏时实现极小极大最优风险性质。虽然用于非常流行的Lasso和弹性网络程序的基于优化的算法可以扩展到数十万个维度，但使用马尔可夫链蒙特卡罗（MCMC）进行计算的马蹄算法仅限于较小数量级的问题。这是由于每一步的高计算成本和时间平均估计量的方差作为维度的函数的增长。我们提出了两个新的MCMC算法计算在这些模型中，有显着提高性能相比，现有的替代品。其中一个算法还近似于一个昂贵的矩阵产品，在高维应用中提供数量级的加速。我们证明保证近似算法的准确性，并表明，逐步减少近似误差链的延伸结果在一个精确的算法。该算法的可扩展性在问题大小为N = 5，000个观察值和p = 50，000个预测值的模拟中得到了说明，并应用于全基因组关联研究，N = 2，267和p = 98，385。实证结果还表明，新算法产生的估计具有较低的均方误差，更好的覆盖范围的区间，并阐明了以前的算法往往错过了在高维，包括双峰的后边缘表示不确定性的协变量属于模型的后验特征。
The horseshoe prior is frequently employed in Bayesian analysis of high-dimensional models, and has been shown to achieve minimax optimal risk properties when the truth is sparse. While optimization-based algorithms for the extremely popular Lasso and elastic net procedures can scale to dimension in the hundreds of thousands, algorithms for the horseshoe that use Markov chain Monte Carlo (MCMC) for computation are limited to problems an order of magnitude smaller. This is due to high computational cost per step and growth of the variance of time-averaging estimators as a function of dimension. We propose two new MCMC algorithms for computation in these models that have significantly improved performance compared to existing alternatives. One of the algorithms also approximates an expensive matrix product to give orders of magnitude speedup in high-dimensional applications. We prove guarantees for the accuracy of the approximate algorithm, and show that gradually decreasing the approximation error as the chain extends results in an exact algorithm. The scalability of the algorithm is illustrated in simulations with problem size as large as N = 5, 000 observations and p = 50, 000 predictors, and an application to a genome-wide association study with N = 2, 267 and p = 98, 385. The empirical results also show that the new algorithm yields estimates with lower mean squared error, intervals with better coverage, and elucidates features of the posterior that were often missed by previous algorithms in high dimensions, including bimodality of posterior marginals indicating uncertainty about which covariates belong in the model.