LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA
LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA
复制标题
DOI：
10.1214/07-aos582
复制
发表时间：
2009-02-01
影响因子：
4.5
通讯作者：
Yu, Bin
中科院分区：
文献类型：
--
作者：
Meinshausen, Nicolai;Yu, Bin
Lasso是一种用于高维数据的正则化和变量选择的有吸引力的技术，其中预测变量p（n）的数量可能远大于样本数量n。然而，最近发现，稀疏模式的Lasso估计只能渐近相同的真实稀疏模式，如果设计矩阵满足所谓的不可表示的条件。在存在高度相关变量的情况下，后一个条件很容易被违反。在这里，我们研究了放松不可表示条件时Lasso估计的行为。即使Lasso不能恢复正确的稀疏模式，我们证明了估计仍然是一致的，在l（2）-范数意义下的固定设计的条件下（a）的数量s（n）的非零分量的向量beta（n）和（B）的最小奇异值的设计矩阵，通过选择小的变量子集。此外，在适当选择平滑参数的情况下，获得了关于l（2）误差的收敛速度结果。在最大和最小稀疏特征值有界的条件下，该速率是最优的。我们的研究结果意味着，以高概率，所有重要的变量被选中。选定的变量集是对原始变量集的有意义的缩减。最后，我们的结果说明了检测密切相邻的频率，天体物理学中遇到的一个问题。
The Lasso is an attractive technique for regularization and variable selection for high-dimensional data, where the number of predictor variables p(n) is potentially much larger than the number of samples n. However, it was recently discovered that the sparsity pattern of the Lasso estimator can only be asymptotically identical to the true sparsity pattern if the design matrix satisfies the so-called irrepresentable condition. The latter condition can easily be violated in the presence of highly correlated variables.Here we examine the behavior of the Lasso estimators if the irrepresentable condition is relaxed. Even though the Lasso cannot recover the correct sparsity pattern, we show that the estimator is still consistent in the l(2)-norm sense for fixed designs under conditions on (a) the number s(n) Of nonzero components of the vector beta(n) and (b) the minimal singular values of design matrices that are induced by selecting small subsets of variables. Furthermore, a rate of convergence result is obtained on the l(2) error with an appropriate choice of the smoothing parameter. The rate is shown to be optimal under the condition of bounded maximal and minimal sparse eigenvalues. Our results imply that, with high probability, all important variables are selected. The set of selected variables is a meaningful reduction on the original set of variables. Finally, our results are illustrated with the detection of closely adjacent frequencies, a problem encountered in astrophysics.