CAREER: Structured Minimax Optimization: Theory, Algorithms, and Applications in Robust Learning

职业：结构化极小极大优化：稳健学习中的理论、算法和应用

• 批准号: 2338846
• 负责人: Aryan Mokhtari
• 金额: $ 66万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-08-01 至 2029-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2338846&HistoricalAwards=false
• 关键词: CAREER; Structured; Minimax; Optimization; Theory

This proposal seeks to extend the mathematical theory of minimax optimization, a very commonly used approach that has played a pivotal role in advancing the fields of information theory, machine learning and signal processing. Notably, there has been a recent surge of interest in minimax optimization due to its critical relevance in artificial intelligence (AI), where it can be used to make deep neural networks more resilient against adversarial disturbances in the underlying distribution of data. While there has been recent progress in enhancing the theory of minimax optimization and its algorithms, a notable gap persists in the applicability of this theory to real-world AI scenarios. Existing algorithms and theory primarily focus on the so-called convex-concave optimization setting, while contemporary learning applications frequently entail minimax problems that do not adhere to this structure and are more complex. This project aims to develop efficient methods for minimax optimization for AI by capitalizing on the unique structure of the AI prediction function. This interdisciplinary project integrates research findings into graduate and undergraduate courses and promotes STEM interest among high school students. The overarching goal of this project is to advance optimization theory and algorithms for robust machine learning model training by investigating their structured minimax optimization problems. The minimization problems in robust learning are structured based on the prediction function model, and we will investigate different types of models, such as neural networks. For each considered prediction function, the minimization component of the minimax problem is nonconvex; nonetheless, they each possess a distinct structure that we meticulously explore in separate thrusts. Moreover, within each thrust, we study different types of structured inner maximization problems motivated by the three applications closely examined in this project: distributionally robust learning, adversarially robust training, and discrete robust learning. The developed algorithms can significantly advance these application areas in terms of computational cost and accuracy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该提案旨在扩展极大极小优化的数学理论，这是一种非常常用的方法，在推进信息论，机器学习和信号处理领域中发挥了关键作用。值得注意的是，最近人们对极大极小优化的兴趣激增，因为它在人工智能（AI）中具有重要的相关性，可以用来使深度神经网络对底层数据分布中的对抗性干扰更具弹性。虽然最近在增强极大极小优化理论及其算法方面取得了进展，但该理论在现实世界AI场景中的适用性仍然存在显着差距。现有的算法和理论主要集中在所谓的凹凸优化设置，而当代的学习应用程序经常需要极大极小问题，不坚持这种结构，更复杂。该项目旨在通过利用AI预测函数的独特结构，开发用于AI的极大极小优化的有效方法。这个跨学科项目将研究成果融入研究生和本科课程，并促进高中生对STEM的兴趣。该项目的总体目标是通过研究结构化极大极小优化问题，推进鲁棒机器学习模型训练的优化理论和算法。鲁棒学习中的最小化问题是基于预测函数模型构造的，我们将研究不同类型的模型，如神经网络。对于每一个考虑的预测函数，极大极小问题的最小化部分是非凸的;尽管如此，它们每个都具有不同的结构，我们在单独的推力中仔细探索。此外，在每一个推力，我们研究不同类型的结构内部最大化问题的动机在这个项目中密切研究的三个应用程序：分布鲁棒学习，adversarially鲁棒训练，离散鲁棒学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。