Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization
Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization
复制标题
DOI：
10.24963/ijcai.2020/201
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Yi Zhou;Zhe Wang-;Kaiyi Ji;Yingbin Liang;V. Tarokh
中科院分区：
文献类型：
--
作者：
Yi Zhou;Zhe Wang-;Kaiyi Ji;Yingbin Liang;V. Tarokh
为了提高近似梯度算法在凸优化问题中的收敛性，人们提出了各种参数重新启动方案。然而，在参数重启下，带动量项的近似梯度算法在非凸优化问题中的收敛性仍然不明显。本文提出了一种新的带动量和参数重启的近似梯度算法来求解非凸非光滑问题。我们的算法被设计为：1）允许采用灵活的参数重新启动计划，覆盖许多现有的; 2）在非凸和非光滑优化中具有全局次线性收敛速度; 3）保证收敛到临界点，并具有各种类型的渐近收敛速度，这取决于非凸和非光滑优化中局部几何的参数化。数值实验证明了算法的收敛性和有效性。
Various types of parameter restart schemes have been proposed for proximal gradient algorithm with momentum to facilitate their convergence in convex optimization. However, under parameter restart, the convergence of proximal gradient algorithm with momentum remains obscure in nonconvex optimization. In this paper, we propose a novel proximal gradient algorithm with momentum and parameter restart for solving nonconvex and nonsmooth problems. Our algorithm is designed to 1) allow for adopting flexible parameter restart schemes that cover many existing ones; 2) have a global sub-linear convergence rate in nonconvex and nonsmooth optimization; and 3) have guaranteed convergence to a critical point and have various types of asymptotic convergence rates depending on the parameterization of local geometry in nonconvex and nonsmooth optimization. Numerical experiments demonstrate the convergence and effectiveness of our proposed algorithm.