A dense initialization for limited-memory quasi-Newton methods
A dense initialization for limited-memory quasi-Newton methods
复制标题
DOI：
10.1007/s10589-019-00112-x
复制
发表时间：
2019-05
影响因子：
2.2
通讯作者：
J. Brust;O. Burdakov;Jennifer B. Erway;Roummel F. Marcia
中科院分区：
文献类型：
--
作者：
J. Brust;O. Burdakov;Jennifer B. Erway;Roummel F. Marcia
我们考虑一类有限内存准牛顿方法的密集初始化。提出的初始化利用基于特征分解的全空间分离为两个互补的子空间，为每个子空间分配不同的初始化参数。这类密集初始化是在有限记忆Broyden-Fletcher-Goldfarb-Shanno （L-BFGS）信任域方法的背景下提出的，该方法利用形状变化范数来定义每个子问题。与传统上使用对角初始化的L-BFGS方法一样，稠密初始化和生成的拟牛顿矩阵序列从未显式形成。CUTEst测试集上的数值实验表明，该初始化方法与形状变化信任域方法在求解一般非凸无约束优化问题时优于其他L-BFGS方法。虽然这种密集初始化是在一种特殊的信任域方法中提出的，但它在更一般的准牛顿信任域和线搜索方法中具有广泛的应用。事实上，这种初始化适用于任何允许紧凑表示的准牛顿更新，特别是Broyden更新类的任何成员。
We consider a family of dense initializations for limited-memory quasi-Newton methods. The proposed initialization exploits an eigendecomposition-based separation of the full space into two complementary subspaces, assigning a different initialization parameter to each subspace. This family of dense initializations is proposed in the context of a limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) trust-region method that makes use of a shape-changing norm to define each subproblem. As with L-BFGS methods that traditionally use diagonal initialization, the dense initialization and the sequence of generated quasi-Newton matrices are never explicitly formed. Numerical experiments on the CUTEst test set suggest that this initialization together with the shape-changing trust-region method outperforms other L-BFGS methods for solving general nonconvex unconstrained optimization problems. While this dense initialization is proposed in the context of a special trust-region method, it has broad applications for more general quasi-Newton trust-region and line search methods. In fact, this initialization is suitable for use with any quasi-Newton update that admits a compact representation and, in particular, any member of the Broyden class of updates.