Model selection properties of forward selection and sequential cross‐validation for high‐dimensional regression
Model selection properties of forward selection and sequential cross‐validation for high‐dimensional regression
复制标题
DOI：
10.1002/cjs.11635
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
J. Wieczorek;Jing Lei
中科院分区：
文献类型：
--
作者：
J. Wieczorek;Jing Lei
前向选择（FS）是线性回归中一种流行的变量选择方法。但是，对协变量个数发散的FS的理论认识仍然有限。我们得到了FS达到模型选择一致性的充分条件。我们的条件是类似的正交匹配追求，但使用不同的参数。当真实模型大小未知时，我们基于交叉验证的顺序变体，推导出具有数据驱动停止规则的FS模型选择一致性的充分条件。作为我们的证明的副产品，我们也有一个尖锐的（充分和几乎必要的）条件的模型选择一致性的“包装器”前向搜索线性回归。我们说明了直觉，并证明我们的方法使用模拟研究和真实的数据集的性能。
Forward selection (FS) is a popular variable selection method for linear regression. But theoretical understanding of FS with a diverging number of covariates is still limited. We derive sufficient conditions for FS to attain model selection consistency. Our conditions are similar to those for orthogonal matching pursuit, but are obtained using a different argument. When the true model size is unknown, we derive sufficient conditions for model selection consistency of FS with a data‐driven stopping rule, based on a sequential variant of cross‐validation. As a byproduct of our proofs, we also have a sharp (sufficient and almost necessary) condition for model selection consistency of “wrapper” forward search for linear regression. We illustrate intuition and demonstrate performance of our methods using simulation studies and real datasets.