Learning Proximal Operators to Discover Multiple Optima
Learning Proximal Operators to Discover Multiple Optima
复制标题
DOI：
--
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Lingxiao Li;Noam Aigerman;Vladimir G. Kim;Jiajin Li;K. Greenewald;M. Yurochkin;J. Solomon
中科院分区：
文献类型：
--
作者：
Lingxiao Li;Noam Aigerman;Vladimir G. Kim;Jiajin Li;K. Greenewald;M. Yurochkin;J. Solomon
寻找非凸优化问题的多个解决方案是一项普遍存在但具有挑战性的任务。大多数过去的算法要么应用来自多个随机初始猜测的单解优化方法，要么使用临时启发式在找到的解附近进行搜索。我们提出了一种端到端的方法来学习一系列训练问题的近端算子，以便通过迭代学习的算子，模拟具有快速收敛的近端点算法，可以从初始猜测中快速获得多个局部最小值。学习到的近端算子可以进一步泛化，以恢复测试时未见问题的多个最优值，从而实现对象检测等应用。我们公式中的关键成分是近端正则化项，它提高了训练损失的凸性：通过应用最近的理论结果，我们表明，对于具有 Lipschitz 梯度的弱凸目标，近端算子的训练以实际程度的超参数化全局收敛。我们进一步提出了多解决方案优化的详尽基准，以证明我们方法的有效性。
Finding multiple solutions of non-convex optimization problems is a ubiquitous yet challenging task. Most past algorithms either apply single-solution optimization methods from multiple random initial guesses or search in the vicinity of found solutions using ad hoc heuristics. We present an end-to-end method to learn the proximal operator of a family of training problems so that multiple local minima can be quickly obtained from initial guesses by iterating the learned operator, emulating the proximal-point algorithm that has fast convergence. The learned proximal operator can be further generalized to recover multiple optima for unseen problems at test time, enabling applications such as object detection. The key ingredient in our formulation is a proximal regularization term, which elevates the convexity of our training loss: by applying recent theoretical results, we show that for weakly-convex objectives with Lipschitz gradients, training of the proximal operator converges globally with a practical degree of over-parameterization. We further present an exhaustive benchmark for multi-solution optimization to demonstrate the effectiveness of our method.