Honest variable selection in linear and logistic regression models via l1 and l1 + l2 penalization
Honest variable selection in linear and logistic regression models via l1 and l1 + l2 penalization
复制标题
DOI：
10.1214/08-ejs287
复制
发表时间：
2008-01-01
影响因子：
1.1
通讯作者：
Bunea, Florentina
中科院分区：
文献类型：
--
作者：
Bunea, Florentina
本文研究了有限样本下l（1）和l（1）+ l（2）型惩罚方案的正确变量选择问题。变量选择的渐近一致性立即从这个分析。我们专注于逻辑和线性回归模型。以下问题是我们的论文的核心：给定置信度1 - delta，在设计矩阵的假设下，对于信号的强度以及调谐参数的值，我们可以在给定的置信度下确定真实的模型？形式上，如果（I）over cap是真实变量集I* 的估计，我们研究了P（（I）over cap = I*）>= 1 - delta的条件，对于给定的样本量n，参数数34和置信度1 - delta。我们发现，在可识别的模型，这两种方法可以恢复系数的大小为1/根n，小的乘法常数和对数因子在M和1/δ。对于变量选择问题，l（1）+ l（2）惩罚优于l（1），对于我们在这里考虑的模型。而前者的估计是唯一的，并变得更加稳定的高度相关的数据矩阵，因为一个增加的调整参数的l（2）部分，太大的增加，在这个参数值可能会妨碍变量的选择。
This paper investigates correct variable selection in finite samples via l(1) and l(1) + l(2) type penalization schemes. The asymptotic consistency of variable selection immediately follows from this analysis. We focus on logistic and linear regression models. The following questions are central to our paper: given a level of confidence 1 - delta, under which assumptions on the design matrix, for which strength of the signal and for what values of the tuning parameters can we identify the true model at the given level of confidence? Formally, if (I) over cap is an estimate of the true variable set I*, we study conditions under which P((I) over cap = I*) >= 1 - delta, for a given sample size n, number of parameters 34 and confidence 1 - delta. We show that in identifiable models, both methods can recover coefficients of size 1/root n, up to small multiplicative constants and logarithmic factors in M and 1/delta. The advantage of the l(1) + l(2) penalization over the l(1) is minor for the variable selection problem, for the models we consider here. Whereas the former estimates are unique, and become more stable for highly correlated data matrices as one increases the tuning parameter of the l(2) part, too large an increase in this parameter value may preclude variable selection.