Greedy Projected Gradient-Newton Method for Sparse Logistic Regression
Greedy Projected Gradient-Newton Method for Sparse Logistic Regression
复制标题
DOI：
10.1109/tnnls.2019.2905261
复制
发表时间：
2020-02
影响因子：
10.4
通讯作者：
Rui Wang;N. Xiu;Chao Zhang
中科院分区：
文献类型：
--
作者：
Rui Wang;N. Xiu;Chao Zhang
稀疏逻辑回归（SLR）是具有稀疏约束的经典逻辑回归模型，广泛应用于神经网络、深度学习和生物信息学等领域的分类和特征选择。在本文中，我们进行了理论分析的存在性和唯一性的SLR，我们提出了一个贪婪的投影梯度牛顿（GPGN）方法求解SLR。GPGN方法是投影梯度法和牛顿法的组合。GPGN方法在求解SLR问题时，不仅获得了优美的理论结果，而且具有显著的数值性能：1）在较弱的条件下，GPGN方法产生的全迭代序列收敛于SLR的全局/局部极小点：2）GPGN方法具有最优支撑集的有限识别性和局部二次收敛性; 3）通过数值实验，与现有的一些求解器相比，GPGN方法具有更高的精度和更快的速度。
Sparse logistic regression (SLR), which is widely used for classification and feature selection in many fields, such as neural networks, deep learning, and bioinformatics, is the classical logistic regression model with sparsity constraints. In this paper, we perform theoretical analysis on the existence and uniqueness of the solution to the SLR, and we propose a greedy projected gradient-Newton (GPGN) method for solving the SLR. The GPGN method is a combination of the projected gradient method and the Newton method. The following characteristics show that the GPGN method achieves not only elegant theoretical results but also a remarkable numerical performance in solving the SLR: 1) the full iterative sequence generated by the GPGN method converges to a global/local minimizer of the SLR under weaker conditions; 2) the GPGN method has the properties of afinite identification for an optimal support set and local quadratic convergence; and 3) the GPGN method achieves higher accuracy and higher speed compared with a number of state-of-the-art solvers according to numerical experiments.