Local Convergence of Generalized Gauss-Newton and Sequential Convex Programming
Local Convergence of Generalized Gauss-Newton and Sequential Convex Programming
复制标题
广义高斯-牛顿和顺序凸规划的局部收敛
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
F. Messerer
中科院分区：
文献类型：
--
作者：
M. Diehl;F. Messerer
本文分析了求解具有凸子结构的无约束非线性优化问题的两种牛顿型算法：广义高斯-牛顿算法（GGN）和序列凸规划算法（SCP）的收敛性。虽然这两种算法是相同的经典高斯-牛顿法的特殊情况下的非线性最小二乘，它们不同时，适用于更一般的凸外函数。在较弱的假设下，我们证明了GGN和SCP具有相同收缩率的局部线性收敛性。收敛或发散率可以表征为满足两个线性矩阵不等式的最小标量。我们进一步表明，在一个给定的局部最小值的不良收敛或发散可以是一个理想的属性的背景下，估计问题的对称似然函数，因为它避免了算法被吸引的统计上不可取的局部极小值。数值例子说明了这两种算法及其收敛性。
We analyze the convergence properties of two Newton-type algorithms for the solution of unconstrained nonlinear optimization problems with convex substructure: Generalized Gauss-Newton (GGN) and Sequential Convex Programming (SCP). While both algorithms are identical to the classical Gauss-Newton method for the special case of nonlinear least squares, they differ when applied to more general convex outer functions. We show under mild assumptions that GGN and SCP have locally linear convergence with the same contraction rate. The convergence or divergence rate can be characterized as the smallest scalar that satisfies two linear matrix inequalities. We further show that bad convergence or divergence at a given local minimum can be a desirable property in the context of estimation problems with symmetric likelihood functions, because it avoids that the algorithm is attracted by statistically undesirable local minima. Both algorithms and their convergence properties are illustrated with a numerical example.