ON THE LIMITED MEMORY BFGS METHOD FOR LARGE-SCALE OPTIMIZATION
ON THE LIMITED MEMORY BFGS METHOD FOR LARGE-SCALE OPTIMIZATION
复制标题
DOI：
10.1007/bf01589116
复制
发表时间：
1989-12-01
影响因子：
2.7
通讯作者：
NOCEDAL, J
中科院分区：
文献类型：
--
作者：
LIU, DC;NOCEDAL, J
本文研究了一种求解大规模优化问题的有限记忆拟牛顿法的数值性能，我们称之为L-BFGS法。我们将其性能与Buckley和LeNir（1985）开发的方法进行了比较，后者结合了BFGS步骤和共轭方向步骤的循环。数值试验表明，L-BFGS方法比Buckley和LeNir方法更快，并且能够更好地利用额外的存储来加速收敛。我们表明，L-BFGS方法可以大大加快通过一个简单的缩放。然后将L-BFGS方法与Griewank和Toint（1982 a）的分块拟牛顿方法进行了比较。结果表明，对于某些问题，分区拟牛顿法明显上级L-BFGS法。然而，我们发现，对于其他问题的L-BFGS方法是非常有竞争力的，由于其低的迭代成本。我们还研究了L-BFGS方法的收敛性，证明了其在一致凸问题上的全局收敛性。
We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.