Practical perspectives on symplectic accelerated optimization
Practical perspectives on symplectic accelerated optimization
复制标题
DOI：
10.1080/10556788.2023.2214837
复制
发表时间：
2023-06-06
影响因子：
2.2
通讯作者：
Leok，Melvin
中科院分区：
文献类型：
--
作者：
Duruisseaux，Valentin;Leok，Melvin
几何数值积分最近被用来设计辛加速优化算法模拟布雷格曼拉格朗日和哈密顿系统的变分框架介绍了Wibisono等人。在本文中，我们讨论了实际的考虑，可以显着提高这些优化算法的计算性能，大大简化了调整过程。特别是，我们研究如何动量重新启动计划改善计算效率和鲁棒性，通过减少振荡的不良影响，并通过使时间自适应多余的调整过程。我们还讨论了时间循环如何帮助避免数值精度引起的不稳定问题，而不会损害算法的计算效率。最后，我们比较了不同几何积分技术的效率和鲁棒性，并研究了算法中不同参数的影响，以在实践中提供信息并简化调整。从本文中出现了辛加速优化算法，其计算效率、稳定性和鲁棒性都得到了提高，并且现在使用和调整实际应用要简单得多。
Geometric numerical integration has recently been exploited to design symplectic accelerated optimization algorithms by simulating the Bregman Lagrangian and Hamiltonian systems from the variational framework introduced by Wibisono et al. In this paper, we discuss practical considerations which can significantly boost the computational performance of these optimization algorithms and considerably simplify the tuning process. In particular, we investigate how momentum restarting schemes ameliorate computational efficiency and robustness by reducing the undesirable effect of oscillations and ease the tuning process by making time-adaptivity superfluous. We also discuss how temporal looping helps avoiding instability issues caused by numerical precision, without harming the computational efficiency of the algorithms. Finally, we compare the efficiency and robustness of different geometric integration techniques and study the effects of the different parameters in the algorithms to inform and simplify tuning in practice. From this paper emerge symplectic accelerated optimization algorithms whose computational efficiency, stability and robustness have been improved, and which are now much simpler to use and tune for practical applications.