Collaborative Research: CIF: Small: New Theory, Algorithms and Applications for Large-Scale Bilevel Optimization

合作研究：CIF：小型：大规模双层优化的新理论、算法和应用

• 批准号: 2311275
• 负责人: Shiqian Ma
• 金额: $ 29.95万
• 依托单位: William Marsh Rice University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311275&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; New

In recent years, the world has witnessed significant progress in optimization for emerging fields, including meta-learning, fine-tuning, automated hyperparameter selection, continual learning, fair batch selection, adversarial learning, and artificial intelligence (AI)-aware communication networks. Problems arising from these fields often exhibit a common nested optimization structure, which has motivated the study of bilevel optimization. However, there are many theoretical and computational challenges in large-scale bilevel optimization problems, e.g., those arising from machine learning on massive amounts of data in high-dimensional feature domains that have manifold constraints. This project will provide a comprehensive study of bilevel optimization theory, algorithms, and applications for large-scale problems. The outcomes of this project will benefit researchers in academia, government labs, and industry aiming to solve large-scale nested optimization problems in science and engineering. New applications in information science, signal processing, communications, statistics, and machine learning will be studied. This project consists of three intertwined thrusts. The first thrust focuses on developing fast and scalable Hessian-free bilevel algorithms with convergence rate guarantees. Specifically, several Hessian-free approaches will be designed and analyzed using methods of fully single-loop momentum, finite-difference matrix-vector estimation, and residual response Jacobian estimation. The second thrust aims to develop primal-dual, primal, and pessimistic bilevel methods, in addition to the analysis of convergence in the difficult case where no unique lower-level solution exists. In the third thrust, the investigators will develop algorithms for solving bilevel problems on non-linear manifolds and analyze the associated convergence of these algorithms. The developed algorithms will be implemented in the context of real-world applications, including fairness-aware machine learning, continual learning, resource allocation over communication networks, hyperparameter selection of principal component analysis, and dictionary learning models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，世界在新兴领域的优化方面取得了重大进展，包括元学习，微调，自动化超参数选择，持续学习，公平的批次选择，对抗性学习和人工智能（AI）感知通信网络。这些领域引起的问题通常表现出一个共同的嵌套优化结构，这激发了双重优化的研究。但是，在大规模的双重优化问题中存在许多理论和计算挑战，例如，在具有多种限制因素的高维特征域中大量数据引起的机器学习引起的挑战。该项目将提供一项全面研究双层优化理论，算法和针对大规模问题的应用。该项目的结果将使学术界，政府实验室和行业的研究人员受益，旨在解决科学和工程学中的大规模嵌套优化问题。将研究信息科学，信号处理，通信，统计和机器学习中的新应用。该项目由三个相互交织的推力组成。第一个推力侧重于开发具有收敛速率保证的快速和可扩展的无HESSIAN双层算法。具体而言，将使用完全单环动量，有限差分矩阵向量估计和剩余响应雅各布估计的方法来设计和分析几种无HESSIAN方法。第二个推力旨在在不存在独特的低级解决方案的困难情况下，开发出原始的双偶，原始和悲观的二聚体方法。在第三个推力中，研究人员将开发算法，以解决非线性歧管上的二聚体问题，并分析这些算法的相关收敛性。开发的算法将在现实世界应用的背景下实施，包括公平感知机器学习，持续学习，对通信网络上的资源分配，主参数的选择主成分分析和词典学习模型。这项奖项反映了NSF的立法使命，并被认为是通过基金会的智力评估和广泛的评估来评估的，并且值得通过评估来进行评估。