PA-GD: On the Convergence of Perturbed Alternating Gradient Descent to Second-Order Stationary Points for Structured Nonconvex Optimization
PA-GD: On the Convergence of Perturbed Alternating Gradient Descent to Second-Order Stationary Points for Structured Nonconvex Optimization
复制标题
PA-GD：结构化非凸优化的扰动交替梯度下降到二阶驻点的收敛性
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Zhengdao Wang
中科院分区：
文献类型：
--
作者：
Songtao Lu;Mingyi Hong;Zhengdao Wang
交替梯度下降(A-GD)是机器学习中一种简单但流行的算法，它使用梯度下降步骤交替地更新两个变量块。本文考虑一类光滑的无约束非凸优化问题，提出了一个全局次线性收敛到二阶平稳点(SOSP)的扰动A-GD(PA-GD)。已有的关于A-GD型算法的分析要么只保证收敛到fi一阶解，要么渐近收敛到二阶解(没有速率)。就我们所知，这是fi第一交替型算法，它需要O(PolyLog(D)/ϵ2)次迭代才能以高概率获得(ϵ，√ϵ)-SOSP，其中PolyLog(D)表示关于问题维度d的对数的多项式。
Alternating gradient descent (A-GD) is a simple but popular algorithm in machine learning, which updates two blocks of variables in an alternating manner using gradient descent steps. In this paper, we consider a smooth unconstrained nonconvex optimization problem, and propose a p erturbed A - GD (PA-GD) which is able to converge (with high probability) to the second-order stationary points (SOSPs) with a global sublinear rate. Existing analysis on A-GD type algorithm either only guarantees convergence to first-order solutions, or converges to second-order solutions asymptotically (without rates). To the best of our knowledge, this is the first alternating type algorithm that takes O ( polylog ( d ) /ϵ 2 ) iterations to achieve an ( ϵ, √ ϵ )-SOSP with high probability, where polylog ( d ) denotes the polynomial of the logarithm with respect to problem dimension d .
影响因子：
2.5
作者：
Zhihui Zhu;Qiuwei Li;Gongguo Tang;M. Wakin
通讯作者：
Zhihui Zhu;Qiuwei Li;Gongguo Tang;M. Wakin
影响因子：
4.300
作者：
Hagar Ibrahim Labouta;Labiba K. El-Khordagui
通讯作者：
Labiba K. El-Khordagui
影响因子：
14.9
作者：
Chen, Yudong;Chi, Yuejie
通讯作者：
Chi, Yuejie