A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem
A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem
复制标题
DOI：
--
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Ruichen Jiang;Nazanin Abolfazli;Aryan Mokhtari;E. Y. Hamedani
中科院分区：
文献类型：
--
作者：
Ruichen Jiang;Nazanin Abolfazli;Aryan Mokhtari;E. Y. Hamedani
在本文中，我们研究了一类二聚体优化问题，也称为简单的双重优化，在此中，我们将平稳的目标函数最小化了另一个凸的约束优化问题的最佳解决方案集。已经开发了几种解决此类问题的迭代方法。 las，它们的收敛保证是对上层物镜的渐近性，或者收敛速度缓慢且亚最佳。为了解决这个问题，在本文中，我们引入了一种新颖的双层优化方法，该方法通过切割平面在局部近似下层问题的解决方案集，然后运行条件梯度更新以减少上层目标。当上层目标是凸面时，我们表明我们的方法需要$ {\ Mathcal {o}}（\ max \ {1/\ epsilon_f，1/\ epsilon_g \}）$迭代才能找到$ \ \ \ \ \ \ \ epsilon_f $ - 最佳目标目标和$ \ epsilon_g $ - 最佳目标目标。此外，当高级目标是非convex时，我们的方法需要$ {\ Mathcal {o}}（\ max \ {1/\ epsilon_f^2,1/（\ epsilon_f \ epsilon_f \ epsilon_g}）查找$（\ epsilon_f，\ epsilon_g）$ - 最佳解决方案。我们还证明，在“较低级别问题的老年人错误约束假设”下，我们证明了更强的融合保证。据我们所知，我们的方法实现了所考虑的一类二聚体问题的最著名迭代复杂性。
In this paper, we study a class of bilevel optimization problems, also known as simple bilevel optimization, where we minimize a smooth objective function over the optimal solution set of another convex constrained optimization problem. Several iterative methods have been developed for tackling this class of problems. Alas, their convergence guarantees are either asymptotic for the upper-level objective, or the convergence rates are slow and sub-optimal. To address this issue, in this paper, we introduce a novel bilevel optimization method that locally approximates the solution set of the lower-level problem via a cutting plane, and then runs a conditional gradient update to decrease the upper-level objective. When the upper-level objective is convex, we show that our method requires ${\mathcal{O}}(\max\{1/\epsilon_f,1/\epsilon_g\})$ iterations to find a solution that is $\epsilon_f$-optimal for the upper-level objective and $\epsilon_g$-optimal for the lower-level objective. Moreover, when the upper-level objective is non-convex, our method requires ${\mathcal{O}}(\max\{1/\epsilon_f^2,1/(\epsilon_f\epsilon_g)\})$ iterations to find an $(\epsilon_f,\epsilon_g)$-optimal solution. We also prove stronger convergence guarantees under the H\"olderian error bound assumption on the lower-level problem. To the best of our knowledge, our method achieves the best-known iteration complexity for the considered class of bilevel problems.