CAREER: Learning Optimization Algorithms from Data: Interpretability, Reliability, and Scalability

职业：从数据中学习优化算法：可解释性、可靠性和可扩展性

• 批准号: 2145346
• 负责人: Zhangyang Wang
• 金额: $ 50万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-15 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145346&HistoricalAwards=false
• 关键词: CAREER; Learning; Optimization; Algorithms; Data

Efficient and scalable optimization algorithms (a.k.a., optimizers) are the cornerstone of almost all computational fields. In many practical applications of optimization, one will repeatedly perform a certain type of optimization tasks over a specific distribution of data. Learning to optimize (L2O) is an emerging paradigm that automatically develops an optimization method (optimizer) by learning from its performance on a set of past optimization tasks. Then on solving new but similar optimization tasks, the learned optimizer can demonstrate many promising benefits including faster convergence and/or better solution quality. As a fast-growing new field, many open challenges remain concerning both L2O's theoretical underpinnings and its practical applicability. In particular, the learned optimizers are often hard to interpret, trust, and scale.The project targets those research gaps and expands to mid-term and long-term research directions pertaining to the foundations of L2O. Specifically, the project proposes a multi-pronged research agenda including: a novel symbolic representation that makes L2O lightweight and more interpretable; a Bayesian L2O modeling framework that can quantify optimizer uncertainty; new customized designs of L2O model architectures and regularizers that can robustly encode problem-specific priors; and a generic amalgamation scheme to bridge L2O training to classical optimizers as teachers. Each thrust addresses a unique aspect of L2O (representation, calibration, model design, and training strategy). Meanwhile, those thrusts are compatible with each other and can be applied together. The proposed efforts synergize cutting-edge technical advances from deep learning, symbolic learning, Bayesian optimization, and meta learning. Successful outcomes are expected to turn L2O into principled science as well as a mature tool for real applications. This project has an integrated plan of result dissemination, education, and outreach. In particular, all new algorithms resulting from the project will be integrated into the Open-L2O software package, developed and maintained by the PI's group.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高效和可扩展的优化算法（又称，优化器）是几乎所有计算领域的基石。在优化的许多实际应用中，人们将在特定的数据分布上重复执行某种类型的优化任务。学习优化（L2 O）是一种新兴的范式，它通过学习过去一组优化任务的性能来自动开发优化方法（优化器）。然后，在解决新的但类似的优化任务时，学习过的优化器可以展示许多有希望的好处，包括更快的收敛和/或更好的解决方案质量。作为一个快速发展的新领域，L2 O的理论基础和实际应用仍然面临着许多挑战。特别是，学习优化器通常很难解释，信任和规模。该项目针对这些研究空白，并扩展到与L2 O基础相关的中长期研究方向。具体来说，该项目提出了一个多管齐下的研究议程，包括：一种新的符号表示，使L2 O轻量级和更可解释;贝叶斯L2 O建模框架，可以量化优化器的不确定性;新的定制设计的L2 O模型架构和正则化器，可以鲁棒地编码特定问题的先验;和一个通用的合并计划，桥梁L2 O培训经典优化教师。每一个重点都涉及L2 O的一个独特方面（表示，校准，模型设计和培训策略）。同时，这些推力是相互兼容的，可以一起应用。拟议的努力协同了深度学习、符号学习、贝叶斯优化和Meta学习等前沿技术的进步。成功的结果有望将L2 O变成有原则的科学以及真实的应用的成熟工具。该项目有一个成果传播、教育和外联的综合计划。特别是，项目产生的所有新算法将被集成到由PI团队开发和维护的Open-L2 O软件包中。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation

• DOI: 10.48550/arxiv.2303.00039
• 发表时间: 2023-02
• 期刊: ArXiv
• 作者: Junjie Yang;Xuxi Chen;Tianlong Chen;Zhangyang Wang;Yitao Liang