Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
复制标题
稀疏先验高维线性回归的变分贝叶斯
DOI：
10.1080/01621459.2020.1847121
复制
发表时间：
2019
影响因子：
3.7
通讯作者：
Botond Szabó
中科院分区：
文献类型：
--
作者：
Kolyan Ray;Botond Szabó
摘要研究了稀疏高维线性回归中贝叶斯模型选择先验的平均场尖峰和平板变分贝叶斯(VB)逼近。在设计矩阵的相容条件下，导出了平均场VB近似的Oracle不等式，表明它以最优速度收敛到稀疏真理，并给出了响应向量的最优预测。对该算法的实验性能进行了研究，结果表明该算法与其他贝叶斯变量选择方法的性能相当。我们还通过数值证明了广泛使用的坐标上升变分推理算法对参数更新顺序高度敏感，从而导致潜在的较差性能。为了缓解这一问题，我们提出了一种新的优先级更新方案，该方案采用数据驱动的更新顺序，并在仿真中获得了更好的性能。变分算法在R包Sparsevb中实现。这篇文章的补充材料可以在网上找到。
Abstract We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived for the mean-field VB approximation, implying that it converges to the sparse truth at the optimal rate and gives optimal prediction of the response vector. The empirical performance of our algorithm is studied, showing that it works comparably well as other state-of-the-art Bayesian variable selection methods. We also numerically demonstrate that the widely used coordinate-ascent variational inference algorithm can be highly sensitive to the parameter updating order, leading to potentially poor performance. To mitigate this, we propose a novel prioritized updating scheme that uses a data-driven updating order and performs better in simulations. The variational algorithm is implemented in the R package sparsevb. Supplementary materials for this article are available online.
影响因子：
2.5
作者：
E. George;V. Ročková
通讯作者：
E. George;V. Ročková
影响因子：
4.5
作者：
Yang, Yun;Pati, Debdeep;Bhattacharya, Anirban
通讯作者：
Bhattacharya, Anirban
影响因子：
4.5
作者：
Zhang, Fengshuo;Gao, Chao
通讯作者：
Gao, Chao