CAREER: Learning Optimization Algorithms from Data: Interpretability, Reliability, and Scalability

高效和可扩展的优化算法（又称优化器）是几乎所有计算领域的基石。在优化的许多实际应用中，人们将在特定的数据分布上重复执行某种类型的优化任务。学习优化（L2O）是一种新兴的范例，它通过学习一组过去优化任务的性能来自动开发优化方法（优化器）。然后，在解决新的但类似的优化任务时，学习的优化器可以展示许多有希望的好处，包括更快的收敛和/或更好的解决方案质量。作为一个快速发展的新兴领域，在理论基础和实际应用方面仍然存在许多开放性的挑战。特别是，学习的优化器通常难以解释、信任和扩展。该项目针对这些研究空白，并扩展到与L2O基础相关的中长期研究方向。具体来说，该项目提出了一个多管齐下的研究议程，包括：一种新颖的符号表示，使L2O轻量化和更具可解释性；可量化优化器不确定性的贝叶斯L2O建模框架；新的定制设计的L2O模型架构和正则化器，可以对特定问题的先验进行鲁棒编码；以及一个通用的合并方案，将L2O培训与经典优化师作为教师联系起来。每个推力处理L2O的一个独特方面（表示、校准、模型设计和训练策略）。同时，这些推力是相互兼容的，可以一起使用。提议的努力将深度学习、符号学习、贝叶斯优化和元学习的前沿技术进步协同起来。预计成功的结果将使L2O成为有原则的科学以及实际应用的成熟工具。这个项目有一个综合的成果传播、教育和推广计划。特别是，该项目产生的所有新算法将集成到Open-L2O软件包中，由PI的小组开发和维护。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Efficient and scalable optimization algorithms (a.k.a., optimizers) are the cornerstone of almost all computational fields. In many practical applications of optimization, one will repeatedly perform a certain type of optimization tasks over a specific distribution of data. Learning to optimize (L2O) is an emerging paradigm that automatically develops an optimization method (optimizer) by learning from its performance on a set of past optimization tasks. Then on solving new but similar optimization tasks, the learned optimizer can demonstrate many promising benefits including faster convergence and/or better solution quality. As a fast-growing new field, many open challenges remain concerning both L2O's theoretical underpinnings and its practical applicability. In particular, the learned optimizers are often hard to interpret, trust, and scale.The project targets those research gaps and expands to mid-term and long-term research directions pertaining to the foundations of L2O. Specifically, the project proposes a multi-pronged research agenda including: a novel symbolic representation that makes L2O lightweight and more interpretable; a Bayesian L2O modeling framework that can quantify optimizer uncertainty; new customized designs of L2O model architectures and regularizers that can robustly encode problem-specific priors; and a generic amalgamation scheme to bridge L2O training to classical optimizers as teachers. Each thrust addresses a unique aspect of L2O (representation, calibration, model design, and training strategy). Meanwhile, those thrusts are compatible with each other and can be applied together. The proposed efforts synergize cutting-edge technical advances from deep learning, symbolic learning, Bayesian optimization, and meta learning. Successful outcomes are expected to turn L2O into principled science as well as a mature tool for real applications. This project has an integrated plan of result dissemination, education, and outreach. In particular, all new algorithms resulting from the project will be integrated into the Open-L2O software package, developed and maintained by the PI's group.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.