One-step sparse estimates in nonconcave penalized likelihood models
One-step sparse estimates in nonconcave penalized likelihood models
复制标题
DOI：
10.1214/009053607000000802
复制
发表时间：
2008-08-01
影响因子：
4.5
通讯作者：
Li, Runze
中科院分区：
文献类型：
--
作者：
Zou, Hui;Li, Runze
范和李通过使用凹惩罚函数的惩罚似然提出了一系列变量选择方法。非凹惩罚似然估计量具有神谕性质，但最大化惩罚似然函数在计算上具有挑战性，因为目标函数不可微且非凹。在本文中，我们针对一大类凹惩罚函数提出了一种基于局部线性近似（LLA）的新统一算法来最大化惩罚似然。建立了LLA算法的收敛性和其他理论性质。LLA算法的一个显著特点是，在每个LLA步骤中，LLA估计量可以自然地采用稀疏表示。因此，我们建议将LLA算法中的一步LLA估计量作为最终估计量。从统计学角度来看，我们表明如果正则化参数选择得当，一步LLA估计量在有良好初始估计量的情况下具有神谕性质。从计算角度来看，一步LLA估计方法极大地降低了最大化非凹惩罚似然的计算成本。我们进行了一些蒙特卡罗模拟来评估一步稀疏估计方法的有限样本性能。结果非常令人鼓舞。
Fan and Li propose a family of variable selection methods via penalized likelihood using concave penalty functions. The nonconcave penalized likelihood estimators enjoy the oracle properties, but maximizing the penalized likelihood function is computationally challenging, because the objective function is nondifferentiable and nonconcave. In this article, we propose a new unified algorithm based on the local linear approximation (LLA) for maximizing the penalized likelihood for a broad class of concave penalty functions. Convergence and other theoretical properties of the LLA algorithm are established. A distinguished feature of the LLA algorithm is that at each LLA step, the LLA estimator can naturally adopt a sparse representation. Thus, we suggest using the one-step LLA estimator from the LLA algorithm as the final estimates. Statistically, we show that if the regularization parameter is appropriately chosen, the one-step LLA estimates enjoy the oracle properties with good initial estimators. Computationally, the one-step LLA estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood. We conduct some Monte Carlo simulation to assess the finite sample performance of the one-step sparse estimation methods. The results are very encouraging.