A STUDY OF ERROR VARIANCE ESTIMATION IN LASSO REGRESSION
A STUDY OF ERROR VARIANCE ESTIMATION IN LASSO REGRESSION
复制标题
DOI：
10.5705/ss.2014.042
复制
发表时间：
2016-01-01
影响因子：
1.4
通讯作者：
Friedman, Jerome
中科院分区：
文献类型：
--
作者：
Reid, Stephen;Tibshirani, Robert;Friedman, Jerome
线性模型在p b> n时的方差估计是一个难题。标准最小二乘估计技术不适用。在文献中已经提出了几个方差估计量，它们都附有渐近结果，证明了在各种假设下的一致性和渐近正态性。然而，我们发现，当真实的潜在信号随着每个元素信号强度的增大而变得不那么稀疏时，大多数这些估计器在有限样本中会遭受很大的偏差。有一个估计器似乎比它在文献中得到的关注更多：基于残差平方和的估计器，使用Lasso系数和自适应选择的正则化参数（通过交叉验证）。在本文中，我们回顾了几种方差估计器，并进行了相当广泛的模拟研究，试图比较它们的有限样本性能。从结果来看，具有自适应选择的正则化参数的方差估计器在广泛的稀疏性和信号强度设置范围内表现良好。最后，提出并发展了一些关于这类估计器的初步理论分析。
Variance estimation in the linear model when p > n is a difficult problem. Standard least squares estimation techniques do not apply. Several variance estimators have been proposed in the literature, all with accompanying asymptotic results proving consistency and asymptotic normality under a variety of assumptions.It is found, however, that most of these estimators suffer large biases in finite samples when true underlying signals become less sparse with larger per element signal strength. One estimator seems to merit more attention than it has received in the literature: a residual sum of squares based estimator using Lasso coefficients with regularisation parameter selected adaptively (via cross-validation).In this paper, we review several variance estimators and perform a reasonably extensive simulation study in an attempt to compare their finite sample performance. It would seem from the results that variance estimators with adaptively chosen regularisation parameters perform admirably over a broad range of sparsity and signal strength settings. Finally, some intial theoretical analyses pertaining to these types of estimators are proposed and developed.