The Analysis of Optimization Algorithms: A Dissipativity Approach
The Analysis of Optimization Algorithms: A Dissipativity Approach
复制标题
DOI：
10.1109/mcs.2022.3157115
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Laurent Lessard
中科院分区：
文献类型：
--
作者：
Laurent Lessard
工程和应用数学中的优化问题通常以迭代的方式解决，通过系统地调整感兴趣的变量直到找到适当的解。控制这些系统调整的迭代算法可以被视为控制系统。在控制系统中，测量输出，并使用反馈来调整输入，以将误差驱动到零。类似地，在迭代算法中，评估优化目标并调整候选解以将其推向最优点。选择一个能很好地解决各种优化问题的算法类似于鲁棒控制器设计。正如耗散性理论可以用来分析控制系统的稳定性一样，它也可以用来分析迭代算法的收敛性。通过定义一个适当的“能量”的概念，消散与算法的每次迭代，算法的收敛性能可以被表征。本文形式化了迭代算法与控制系统之间的联系，并通过实例展示了如何使用耗散性理论来分析许多类优化算法的性能。这种控制理论的观点，使选择和调整的优化算法，以自动化和系统的方式进行。
Optimization problems in engineering and applied mathematics are typically solved in an iterative fashion, by systematically adjusting the variables of interest until an adequate solution is found. The iterative algorithms that govern these systematic adjustments can be viewed as a control system. In control systems, the output is measured and the input is adjusted using feedback to drive the error to zero. Similarly, in iterative algorithms, the optimization objective is evaluated and the candidate solution is adjusted to drive it toward the optimal point. Choosing an algorithm that works well for a variety of optimization problems is akin to robust controller design. Just as dissipativity theory can be used to analyze the stability properties of control systems, it can also be used to analyze the convergence properties of iterative algorithms. By defining an appropriate notion of “energy” that dissipates with every iteration of the algorithm, the convergence properties of the algorithm can be characterized. This article formalizes the connection between iterative algorithms and control systems and shows through examples how dissipativity theory can be used to analyze the performance of many classes of optimization algorithms. This control-theoretic viewpoint enables the selection and tuning of optimization algorithms to be performed in an automated and systematic way.