Lasso Regression: Estimation and Shrinkage via Limit of Gibbs Sampling
Lasso Regression: Estimation and Shrinkage via Limit of Gibbs Sampling
复制标题
DOI：
--
复制
发表时间：
2014-01
期刊：
影响因子：
--
通讯作者：
B. Rajaratnam;S. Roberts;Doug Sparks;Onkar Dalal
中科院分区：
文献类型：
--
作者：
B. Rajaratnam;S. Roberts;Doug Sparks;Onkar Dalal
套索的应用是支持在高维设置，只有少数的回归系数被认为是非零的。此外，高维Lasso估计的统计性质通常是在预测变量之间的相关性有界的假设下证明的。在这种情况下，坐标方式的方法，最常见的手段计算套索解决方案，以及在存在低到中等的多重共线性。然而，随着稀疏性的降低和多重共线性的增加，坐标算法的计算速度降低。出于这些限制，我们提出了新的“确定性贝叶斯套索”算法计算套索解决方案。该算法是通过考虑限制版本的贝叶斯套索。确定性贝叶斯套索的性能随着稀疏性的降低和多重共线性的增加而提高，并且可以大幅提高计算速度。一个严格的理论分析表明，（1）确定性贝叶斯套索算法收敛到套索解，（2）它导致了一个表示的套索估计，显示它如何实现两种类型的收缩同时$\ell_1$和$\ell_2$。还提供了与其他算法的连接。确定性贝叶斯套索算法的好处，然后说明模拟和真实的数据。
The application of the lasso is espoused in high-dimensional settings where only a small number of the regression coefficients are believed to be nonzero. Moreover, statistical properties of high-dimensional lasso estimators are often proved under the assumption that the correlation between the predictors is bounded. In this vein, coordinatewise methods, the most common means of computing the lasso solution, work well in the presence of low to moderate multicollinearity. The computational speed of coordinatewise algorithms degrades however as sparsity decreases and multicollinearity increases. Motivated by these limitations, we propose the novel "Deterministic Bayesian Lasso" algorithm for computing the lasso solution. This algorithm is developed by considering a limiting version of the Bayesian lasso. The performance of the Deterministic Bayesian Lasso improves as sparsity decreases and multicollinearity increases, and can offer substantial increases in computational speed. A rigorous theoretical analysis demonstrates that (1) the Deterministic Bayesian Lasso algorithm converges to the lasso solution, and (2) it leads to a representation of the lasso estimator which shows how it achieves both $\ell_1$ and $\ell_2$ types of shrinkage simultaneously. Connections to other algorithms are also provided. The benefits of the Deterministic Bayesian Lasso algorithm are then illustrated on simulated and real data.