HESSIAN MATRIX VS. GAUSS-NEWTON HESSIAN MATRIX
HESSIAN MATRIX VS. GAUSS-NEWTON HESSIAN MATRIX
复制标题
DOI：
10.1137/100799988
复制
发表时间：
2011-01-01
影响因子：
2.9
通讯作者：
Chen, Pei
中科院分区：
文献类型：
--
作者：
Chen, Pei
在这篇文章中，我们研究了高斯-牛顿-海森矩阵如何影响牛顿型方法的收敛范围。虽然牛顿算法在理论上优于高斯-牛顿算法和Levenberg MarQuardt(LM)方法，但在实际应用中，在求解非线性最小二乘问题时，往往更倾向于使用LM方法。本文从理论上分析了高斯-牛顿-海森矩阵的优越性。证明了高斯-牛顿逼近函数是唯一保持原目标函数临界性质的非负凸二次逼近函数：取(n-1)维流形(或仿射子空间)上零点的最小值。由于这一性质，高斯-牛顿近似不会改变原问题的零点结构，这解释了为什么高斯-牛顿-海森矩阵是非线性最小二乘问题的首选，特别是当初始点远离解的时候。
In this paper, we investigate how the Gauss-Newton Hessian matrix affects the basin of convergence in Newton-type methods. Although the Newton algorithm is theoretically superior to the Gauss-Newton algorithm and the Levenberg Marquardt (LM) method as far as their asymptotic convergence rate is concerned, the LM method is often preferred in nonlinear least squares problems in practice. This paper presents a theoretical analysis of the advantage of the Gauss-Newton Hessian matrix. It is proved that the Gauss-Newton approximation function is the only nonnegative convex quadratic approximation that retains a critical property of the original objective function: taking the minimal value of zero on an (n - 1)-dimensional manifold (or affine subspace). Due to this property, the Gauss-Newton approximation does not change the zero-on-(n - 1)-D "structure" of the original problem, explaining the reason why the Gauss-Newton Hessian matrix is preferred for nonlinear least squares problems, especially when the initial point is far from the solution.