Variable selection using MM algorithms
Variable selection using MM algorithms
复制标题
DOI：
10.1214/009053605000000200
复制
发表时间：
2005-08-01
影响因子：
4.5
通讯作者：
Li, RZ
中科院分区：
文献类型：
--
作者：
Hunter, DR;Li, RZ
变量选择是高维统计建模的基础。许多变量选择技术可以通过使用各种惩罚函数的最大惩罚似然来实现。优化惩罚似然函数通常具有挑战性，因为它可能是不可微和/或非凹的。本文提出了一类新的算法，寻找一个最大化的惩罚似然的一个广泛的惩罚函数。这些算法通过稍微扰动罚函数来使其可微，然后使用最小化-最大化（MM）算法来优化该可微函数。MM算法是有用的扩展，著名的EM算法类，一个事实，使我们能够分析的局部和全局收敛性的算法使用的一些技术采用EM算法。特别是，我们证明，当我们的MM算法收敛，他们必须收敛到一个理想的点，我们还讨论了条件下，这种收敛可以得到保证。我们利用这些算法的牛顿-拉夫逊方面，提出了一个三明治估计的标准误差的估计。我们的方法在数值试验中表现良好。
Variable selection is fundamental to high-dimensional statistical modeling. Many variable selection techniques may be implemented by maximum penalized likelihood using various penalty functions. Optimizing the penalized likelihood function is often challenging because it may be nondifferentiable and/or nonconcave. This article proposes a new class of algorithms for finding a maximizer of the penalized likelihood for a broad class of penalty functions. These algorithms operate by perturbing the penalty function slightly to render it differentiable, then optimizing this differentiable function using a minorize-maximize (MM) algorithm. MM algorithms are useful extensions of the well-known class of EM algorithms, a fact that allows us to analyze the local and global convergence of the proposed algorithm using some of the techniques employed for EM algorithms. In particular, we prove that when our MM algorithms converge, they must converge to a desirable point; we also discuss conditions under which this convergence may be guaranteed. We exploit the Newton-Raphson-like aspect of these algorithms to propose a sandwich estimator for the standard errors of the estimators. Our method performs well in numerical tests.