When is the first spurious variable selected by sequential regression procedures?
When is the first spurious variable selected by sequential regression procedures?
复制标题
序贯回归程序何时选择第一个虚假变量？
DOI：
10.1093/biomet/asy032
复制
发表时间：
2017
期刊：
影响因子：
2.7
通讯作者：
Weijie J. Su
中科院分区：
文献类型：
--
作者：
Weijie J. Su
应用统计学家使用序贯回归程序来产生解释变量的排名，并且在变量之间的低相关性和强真实效应大小的设置中，期望排名最高的变量与响应真正相关。然而，在一定的稀疏性水平的制度，顺序程序的三个例子-向前逐步，套索，最小角度回归-被证明包括第一个虚假的变量出乎意料地早。我们推导出一个严格的，尖锐的预测，这三个程序的第一个假变量的排名，证明了第一个假变量发生得越来越早，因为回归系数变得越来越密集。对于统计独立的高斯随机设计和任意大幅度的真实效应，这种违反直觉的现象仍然存在。我们通过识别根本原因来更好地理解这种现象，然后利用这些见解来引入一个简单的可视化工具，称为双排名图，以改进顺序方法。作为这些发现的副产品，我们获得了第一个可证明的结果，证明了套索和最小角度回归在早期阶段的解决方案路径超越正交设计之间的确切等价性。这种等价性可以无缝地继承关于套索到最小角度回归的许多重要模型选择结果。
Applied statisticians use sequential regression procedures to produce a ranking of explanatory variables and, in settings of low correlations between variables and strong true effect sizes, expect that variables at the very top of this ranking are truly relevant to the response. In a regime of certain sparsity levels, however, three examples of sequential procedures--forward stepwise, the lasso, and least angle regression--are shown to include the first spurious variable unexpectedly early. We derive a rigorous, sharp prediction of the rank of the first spurious variable for these three procedures, demonstrating that the first spurious variable occurs earlier and earlier as the regression coefficients become denser. This counterintuitive phenomenon persists for statistically independent Gaussian random designs and an arbitrarily large magnitude of the true effects. We gain a better understanding of the phenomenon by identifying the underlying cause and then leverage the insights to introduce a simple visualization tool termed the double-ranking diagram to improve on sequential methods. As a byproduct of these findings, we obtain the first provable result certifying the exact equivalence between the lasso and least angle regression in the early stages of solution paths beyond orthogonal designs. This equivalence can seamlessly carry over many important model selection results concerning the lasso to least angle regression.
影响因子：
4.5
作者：
Lockhart R;Taylor J;Tibshirani RJ;Tibshirani R
通讯作者：
Tibshirani R