A Two-Step Fixed-Point Proximity Algorithm for a Class of Non-differentiable Optimization Models in Machine Learning
A Two-Step Fixed-Point Proximity Algorithm for a Class of Non-differentiable Optimization Models in Machine Learning
复制标题
DOI：
10.1007/s10915-019-01045-7
复制
发表时间：
2019-09
影响因子：
2.5
通讯作者：
Zheng Li;Guohui Song;Yuesheng Xu
中科院分区：
文献类型：
--
作者：
Zheng Li;Guohui Song;Yuesheng Xu
稀疏学习模型在许多应用领域都很流行。稀疏学习模型中的目标函数通常是非光滑的，这使得数值求解变得困难。针对一类基于稀疏学习的不可微优化模型，提出了一种快速收敛的两步迭代算法。为了克服模型的不可微性的困难，我们首先提出了他们的解决方案的特点，涉及到的功能出现在目标函数的邻近算子的映射的不动点。然后我们引入两步定点算法来计算解。我们建立了建议的两步迭代方案的收敛性结果，并将其与交替方向乘子法（ADMM）进行了比较。特别是，我们得到具体的两步迭代算法的三个模型在机器学习：SVM分类，SVM回归，和SVM分类与组LASSO正则化。在人工数据集和基准数据集上的数值实验表明，该算法在计算时间和内存开销方面优于ADMM和线性规划方法.
Sparse learning models are popular in many application areas. Objective functions in sparse learning models are usually non-smooth, which makes it difficult to solve them numerically. We develop a fast and convergent two-step iteration scheme for solving a class of non-differentiable optimization models motivated from sparse learning. To overcome the difficulty of the non-differentiability of the models, we first present characterizations of their solutions as fixed-points of mappings involving the proximity operators of the functions appearing in the objective functions. We then introduce a two-step fixed-point algorithm to compute the solutions. We establish convergence results of the proposed two-step iteration scheme and compare it with the alternating direction method of multipliers (ADMM). In particular, we derive specific two-step iteration algorithms for three models in machine learning:-SVM classification,-SVM regression, and the SVM classification with the group LASSO regularizer. Numerical experiments with some synthetic datasets and some benchmark datasets show that the proposed algorithm outperforms ADMM and the linear programming method in computational time and memory storage costs.