COORDINATE DESCENT ALGORITHMS FOR LASSO PENALIZED REGRESSION
COORDINATE DESCENT ALGORITHMS FOR LASSO PENALIZED REGRESSION
复制标题
DOI：
10.1214/07-aoas147
复制
发表时间：
2008-03-01
影响因子：
1.8
通讯作者：
Lange, Kenneth
中科院分区：
文献类型：
--
作者：
Wu, Tong Tong;Lange, Kenneth
施加套索惩罚将参数估计缩小到零并执行连续模型选择。 Lasso 惩罚回归能够处理预测变量数量远远超过案例数量的线性回归问题。本文测试了两种非常快的算法，用于使用套索惩罚来估计回归系数。先前已知的 l(2) 算法基于循环坐标下降。我们的新 C 算法基于油贪婪坐标下降和普通 l(1) 回归的 Edgeworth 算法。每个算法都依赖于一个可以通过交叉验证选择的调整常数。在某些回归问题中，很自然地对参数进行分组并对参数进行分组惩罚，而不是单独进行。如果组惩罚与组参数的欧几里得诺林成正比，则可以使用套索惩罚来最大化范数并将参数估计减少到 l(2) 回归。因此，现有算法可以扩展到新的设置。所讨论的每种算法都通过模拟数据或真实数据或两者进行测试。附录证明了 l(2) 算法的贪婪形式收敛于目标函数的最小值。
Imposition of a lasso penalty shrinks parameter estimates toward zero and performs continuous model selection. Lasso penalized regression is capable of handling linear regression problems where the number of predictors far exceeds the number of cases. This paper tests two exceptionally fast algorithms for estimating regression coefficients with a lasso penalty. The previously known l(2) algorithm is based on cyclic coordinate descent. Our new C, algorithm is based oil greedy coordinate descent and Edgeworth's algorithm for ordinary l(1) regression. Each algorithm relies on a tuning constant that can be chosen by cross-validation. In some regression problems it is natural to group parameters and penalize parameters group by group rather than separately. If the group penalty is proportional to the Euclidean norin of the parameters of the group, then it is possible to majorize the norm and reduce parameter estimation to l(2) regression with a lasso penalty. Thus, the existing algorithm can be extended to novel settings. Each of the algorithms discussed is tested via either simulated or real data or both. The Appendix proves that a greedy form of the l(2) algorithm converges to the minimum value of the objective function.