On Model Selection Consistency of Lasso
On Model Selection Consistency of Lasso
复制标题
DOI：
--
复制
发表时间：
2006-12
期刊：
影响因子：
--
通讯作者：
P. Zhao;Bin Yu
中科院分区：
文献类型：
--
作者：
P. Zhao;Bin Yu
在科学和社会科学中，统计模型的稀疏性或简约性对于它们的正确解释至关重要。模型选择是寻找此类模型的常用方法，但通常涉及计算量很大的组合搜索。Lasso （Tibshirani, 1996）现在被用作模型选择的计算上可行的替代方法。因此，研究Lasso对模型选择具有重要意义。本文证明了当样本量n变大时，Lasso在经典固定p和大p情况下选择真模型的几乎充分必要条件，我们称之为不可表示条件。基于这些结果，给出了在实践中可验证的充分条件，与以往的工作相关联，并有助于Lasso在特征选择和稀疏表示方面的应用。这个主要依赖于预测变量协方差的不可表示条件表明，当且（几乎）仅当不在真实模型中的预测因子被真实模型中的预测因子“不可表示”（在某种意义上需要澄清）时，Lasso一致地选择了真实模型。此外，还进行了模拟，以提供对这一结果的见解和理解。
Sparsity or parsimony of statistical models is crucial for their proper interpretations, as in sciences and social sciences. Model selection is a commonly used method to find such models, but usually involves a computationally heavy combinatorial search. Lasso (Tibshirani, 1996) is now being used as a computationally feasible alternative to model selection. Therefore it is important to study Lasso for model selection purposes. In this paper, we prove that a single condition, which we call the Irrepresentable Condition, is almost necessary and sufficient for Lasso to select the true model both in the classical fixed p setting and in the large p setting as the sample size n gets large. Based on these results, sufficient conditions that are verifiable in practice are given to relate to previous works and help applications of Lasso for feature selection and sparse representation. This Irrepresentable Condition, which depends mainly on the covariance of the predictor variables, states that Lasso selects the true model consistently if and (almost) only if the predictors that are not in the true model are "irrepresentable" (in a sense to be clarified) by predictors that are in the true model. Furthermore, simulations are carried out to provide insights and understanding of this result.