Catalyst for Gradient-based Nonconvex Optimization
Catalyst for Gradient-based Nonconvex Optimization
复制标题
DOI：
--
复制
发表时间：
2018-03
期刊：
影响因子：
--
通讯作者：
C. Paquette;Hongzhou Lin;D. Drusvyatskiy;J. Mairal;Zaïd Harchaoui
中科院分区：
文献类型：
--
作者：
C. Paquette;Hongzhou Lin;D. Drusvyatskiy;J. Mairal;Zaïd Harchaoui
我们介绍了一个通用的计划来解决非凸优化问题，使用基于梯度的算法最初设计的凸函数最小化。即使这些方法最初可能需要凸性来操作，所提出的方法允许人们使用它们，而无需假设任何关于目标凸性的知识。在一般情况下，该计划是保证产生一个固定点与典型的一阶方法的最坏情况下的效率，当目标变成凸的，它自动加速在Nesterov的意义上，并实现函数值的近最优收敛速度。最后，我们通过将我们的方法应用于增量算法，如SVRG和佐贺稀疏矩阵分解和学习神经网络，获得了有前途的实验结果。
We introduce a generic scheme to solve non-convex optimization problems using gradient-based algorithms originally designed for minimizing convex functions. Even though these methods may originally require convexity to operate, the proposed approach allows one to use them without assuming any knowledge about the convexity of the objective. In general, the scheme is guaranteed to produce a stationary point with a worst-case efficiency typical of first-order methods, and when the objective turns out to be convex, it automatically accelerates in the sense of Nesterov and achieves near-optimal convergence rate in function values. We conclude the paper by showing promising experimental results obtained by applying our approach to incremental algorithms such as SVRG and SAGA for sparse matrix factorization and for learning neural networks.