CAREER: Structured Minimax Optimization: Theory, Algorithms, and Applications in Robust Learning

这一建议旨在扩展极小极大优化的数学理论，这是一种非常常用的方法，在推动信息论、机器学习和信号处理领域发挥了关键作用。值得注意的是，最近人们对极小极大优化的兴趣激增，因为它在人工智能(AI)中具有关键的相关性，在人工智能中，它可以用来使深度神经网络对底层数据分布中的对手干扰具有更强的弹性。虽然最近在增强极小极大优化理论及其算法方面取得了进展，但在该理论对现实世界人工智能场景的适用性方面仍然存在一个显著的差距。现有的算法和理论主要集中在所谓的凸凹优化设置上，而当代的学习应用经常涉及不遵守这种结构且更复杂的极小极大问题。该项目旨在通过利用人工智能预测函数的独特结构来开发高效的人工智能极小极大优化方法。这个跨学科的项目将研究成果整合到研究生和本科课程中，并提高高中生对STEM的兴趣。这个项目的首要目标是通过研究机器学习模型的结构化极小极大优化问题，提出用于稳健机器学习模型训练的优化理论和算法。稳健学习中的最小化问题是基于预测函数模型构造的，我们将研究不同类型的模型，如神经网络。对于每个考虑的预测函数，极小极大问题的最小化部分是非凸的；尽管如此，它们每个都具有不同的结构，我们在单独的推力中仔细地探索这些结构。此外，在每个推力中，我们研究了不同类型的结构化内部最大化问题，这些问题是由本项目中密切研究的三个应用程序所驱动的：分布式稳健学习、对抗性稳健训练和离散稳健学习。开发的算法可以在计算成本和准确性方面显著推进这些应用领域。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This proposal seeks to extend the mathematical theory of minimax optimization, a very commonly used approach that has played a pivotal role in advancing the fields of information theory, machine learning and signal processing. Notably, there has been a recent surge of interest in minimax optimization due to its critical relevance in artificial intelligence (AI), where it can be used to make deep neural networks more resilient against adversarial disturbances in the underlying distribution of data. While there has been recent progress in enhancing the theory of minimax optimization and its algorithms, a notable gap persists in the applicability of this theory to real-world AI scenarios. Existing algorithms and theory primarily focus on the so-called convex-concave optimization setting, while contemporary learning applications frequently entail minimax problems that do not adhere to this structure and are more complex. This project aims to develop efficient methods for minimax optimization for AI by capitalizing on the unique structure of the AI prediction function. This interdisciplinary project integrates research findings into graduate and undergraduate courses and promotes STEM interest among high school students. The overarching goal of this project is to advance optimization theory and algorithms for robust machine learning model training by investigating their structured minimax optimization problems. The minimization problems in robust learning are structured based on the prediction function model, and we will investigate different types of models, such as neural networks. For each considered prediction function, the minimization component of the minimax problem is nonconvex; nonetheless, they each possess a distinct structure that we meticulously explore in separate thrusts. Moreover, within each thrust, we study different types of structured inner maximization problems motivated by the three applications closely examined in this project: distributionally robust learning, adversarially robust training, and discrete robust learning. The developed algorithms can significantly advance these application areas in terms of computational cost and accuracy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.