One-step Sparse Estimates in Nonconcave Penalized Likelihood Models.
One-step Sparse Estimates in Nonconcave Penalized Likelihood Models.
复制标题
DOI：
10.1214/07-aos0316rej
复制
发表时间：
2008-08
影响因子：
4.5
通讯作者：
H. Zou;Runze Li
中科院分区：
文献类型：
--
作者：
H. Zou;Runze Li
Fan & Li（2001）提出了一种基于凹罚函数的惩罚似然的变量选择方法。非凹惩罚似然估计具有预言性质，但由于目标函数是不可微非凹的，因此最大化惩罚似然函数在计算上具有挑战性.在这篇文章中，我们提出了一个新的统一算法的基础上的局部线性近似（LLA）的最大化惩罚似然的凹罚函数的广泛的类。收敛性和其他理论性质的LLA算法的建立。LLA算法的一个显著特征是，在每个LLA步骤中，LLA估计器可以自然地采用稀疏表示。因此，我们建议使用一步LLA估计的LLA算法作为最终的估计。在统计上，我们证明了如果正则化参数选择适当，一步LLA估计具有良好的初始估计的预言性质。在计算上，一步LLA估计方法大大减少了最大化非凹惩罚似然的计算成本。我们进行了一些Monte Carlo模拟来评估一步稀疏估计方法的有限样本性能。结果非常令人鼓舞。
Fan & Li (2001) propose a family of variable selection methods via penalized likelihood using concave penalty functions. The nonconcave penalized likelihood estimators enjoy the oracle properties, but maximizing the penalized likelihood function is computationally challenging, because the objective function is nondifferentiable and nonconcave. In this article we propose a new unified algorithm based on the local linear approximation (LLA) for maximizing the penalized likelihood for a broad class of concave penalty functions. Convergence and other theoretical properties of the LLA algorithm are established. A distinguished feature of the LLA algorithm is that at each LLA step, the LLA estimator can naturally adopt a sparse representation. Thus we suggest using the one-step LLA estimator from the LLA algorithm as the final estimates. Statistically, we show that if the regularization parameter is appropriately chosen, the one-step LLA estimates enjoy the oracle properties with good initial estimators. Computationally, the one-step LLA estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood. We conduct some Monte Carlo simulation to assess the finite sample performance of the one-step sparse estimation methods. The results are very encouraging.