Improvement of the Nelder-Mead method using Direct Inversion in Iterative Subspace
Improvement of the Nelder-Mead method using Direct Inversion in Iterative Subspace
复制标题
DOI：
10.1007/s11081-021-09620-4
复制
发表时间：
2021-03
影响因子：
2.1
通讯作者：
Haru Kitaoka;K. Amano;Naoya Nishi;T. Sakka
中科院分区：
文献类型：
--
作者：
Haru Kitaoka;K. Amano;Naoya Nishi;T. Sakka
Nelder-Mead （NM）方法具有快速收敛和鲁棒性，是一种广受欢迎的无导数优化算法。然而，众所周知，该方法往往不能收敛或花费很长时间进行大规模优化。在本研究中，利用迭代子空间（DIIS）的直接反演对纳米算法进行了改进。DIIS是一种加速优化方法的技术，从已知解的线性组合中推断出更好的中间解。我们使用不同维度的单峰测试函数比较了新方法（NM- diis）和传统NM方法的运行时间。当目标函数的维数较高时，NM- diis方法的平均效果优于原始NM。应用DIIS后，NM方法中运行时分布的长尾消失了。DIIS也在准梯度方法中实现，准梯度方法是Pham等人开发的NM方法的改进版本。工业信息学报，7(2011):592。组合方法在上凸测试函数中也表现良好。本研究提出了一种实用的优化策略，证明了DIIS的通用性。
The Nelder-Mead (NM) method is a popular derivative-free optimization algorithm owing to its fast convergence and robustness. However, it is known that the method often fails to converge or costs a long time for a large-scale optimization. In the present study, the NM method has been improved using direct inversion in iterative subspace (DIIS). DIIS is a technique to accelerate an optimization method, extrapolating a better intermediate solution from linear-combination of the known ones. We compared runtimes of the new method (NM-DIIS) and the conventional NM method using unimodal test functions with various dimensions. The NM-DIIS method showed better results than the original NM on average when the dimension of the objective function is high. Long tails of the runtime distributions in the NM method have disappeared when DIIS was applied. DIIS has also been implemented in the quasi-gradient method, which is an improved version of the NM method developed by Pham et al. [IEEE Trans. Ind. Informatics, 7 (2011) 592]. The combined method also performed well especially in an upwardly convex test function. The present study proposes a practical optimization strategy and proves the versatility of DIIS.