Learning to Optimize: A Primer and A Benchmark
Learning to Optimize: A Primer and A Benchmark
复制标题
DOI：
--
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Tianlong Chen;Xiaohan Chen;Wuyang Chen;Howard Heaton;Jialin Liu;Zhangyang Wang;W. Yin
中科院分区：
文献类型：
--
作者：
Tianlong Chen;Xiaohan Chen;Wuyang Chen;Howard Heaton;Jialin Liu;Zhangyang Wang;W. Yin
学习优化（L2O）是一种新兴方法，它利用机器学习开发优化方法，旨在减少手工工程的艰辛迭代。它根据其在一组培训问题上的性能来自动化优化方法的设计。该数据驱动的过程生成的方法可以有效地解决与培训中类似的问题。相比之下，优化方法的典型和传统设计是理论驱动的，因此它们可以在理论指定的问题类别中获得绩效保证。差异使L2O适用于在特定数据分布上反复解决某种类型的优化问题，而通常会出于分布外问题而失败。 L2O的实用性取决于目标优化的类型，学习方法的选择结构以及训练程序。这个新的范式促使一个研究人员社区探索L2O并报告他们的发现。本文有望成为L2O进行连续优化的首个综合调查和基准。我们设置分类法，对现有作品和研究方向进行分类，展示见解并确定开放的挑战。我们还针对一些但代表性的优化问题对许多现有的L2O方法进行了基准测试。为了可重现的研究和公平的基准测试目的，我们在https://github.com/vita-group/open-l2o中发布了软件实施和数据。
Learning to optimize (L2O) is an emerging approach that leverages machine learning to develop optimization methods, aiming at reducing the laborious iterations of hand engineering. It automates the design of an optimization method based on its performance on a set of training problems. This data-driven procedure generates methods that can efficiently solve problems similar to those in the training. In sharp contrast, the typical and traditional designs of optimization methods are theory-driven, so they obtain performance guarantees over the classes of problems specified by the theory. The difference makes L2O suitable for repeatedly solving a certain type of optimization problems over a specific distribution of data, while it typically fails on out-of-distribution problems. The practicality of L2O depends on the type of target optimization, the chosen architecture of the method to learn, and the training procedure. This new paradigm has motivated a community of researchers to explore L2O and report their findings. This article is poised to be the first comprehensive survey and benchmark of L2O for continuous optimization. We set up taxonomies, categorize existing works and research directions, present insights, and identify open challenges. We also benchmarked many existing L2O approaches on a few but representative optimization problems. For reproducible research and fair benchmarking purposes, we released our software implementation and data in the package Open-L2O at https://github.com/VITA-Group/Open-L2O.