AF: Small: Bridging the Past and Present of Continuous Optimization for Learning

机器学习(ML)正被用来推动人工智能在科学和社会的许多领域的应用。最终，ML问题必须使用统计和数学技术提炼成可以使用计算算法解决的问题。在过去的25年里，ML在很大程度上依赖于优化领域来提供丰富的技术来描述和解决ML问题。事实上，需要优化解决的ML问题的复杂性和难度都在继续增长。优化工具已经被应用于其中一些现代的ML问题，但通常以缺乏对其性能的理论保证的方式。这个项目旨在开发新的、强大的、原则性的优化方法来解决这些更复杂的现代ML问题。该项目将研究对当前ML研究至关重要的几个优化领域，为这些领域开发新的算法技术和新的理论。优先事项包括发现利用各种ML问题特征的结构的算法，例如稀疏性，以及发展理论分析来说明实际算法的计算性能，包括存在非凸性时的有限时间样本复杂性界限。该项目的三个具体项目包括：(I)开发考虑稀疏性或正则性的凸凹最小-最大问题的算法和分析技术；(Ii)开发用于求解具有非凸函数和随机预言的非线性规划的理论基础算法，其动机是约束神经网络中出现的问题、具有公平性约束的问题和分布稳健优化；以及(Iii)促进优化在强化学习中的使用，例如利用第一个推力中发展的原始-对偶技术，并扩展策略梯度方法的理论，以解决实际实施中不可避免的不准确问题。该项目的研究议程有经典优化的基础，以及优化、控制、学习理论和统计学方面的最新发展。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) is being used to drive applications of artificial intelligence in many areas of science and society. Ultimately, ML problems must be distilled using statistical and mathematical techniques into problems that can be solved using computational algorithms. For the past 25 years, ML has depended heavily on the field of optimization to provide a wealth of techniques to formulate and solve ML problems. Indeed, the ML problems that optimization is called on to solve continue to grow in complexity and difficulty. Optimization tools have been applied to some of these modern ML problems, but often in ways that lack theoretical guarantees on their performance. This project aims to develop new, powerful, principled optimization approaches for solving these more complex modern ML problems. The project will study several areas of optimization that are critical to current research in ML, developing new algorithmic techniques and new theory for these areas. Priorities include the discovery of algorithms that leverage the structures that characterize various ML problems, such as sparsity, and the development of theoretical analysis to illuminate the computational performance of practical algorithms, including finite-time sample complexity bounds in the presence of nonconvexity. Three specific thrusts of the project include (i) the development of algorithms and analysis techniques for convex-concave min-max problems that take into account sparsity or regularity properties; (ii) the development of theoretically grounded algorithms for solving nonlinear programs with nonconvex functions and stochastic oracles, motivated by problems arising in constrained neural networks, problems with fairness constraints, and distributionally robust optimization; and (iii) advancing the use of optimization in reinforcement learning, such as making use of the primal-dual techniques developed in the first thrust and extending the theory of policy gradient methods to account for the inexactness that inevitably arises in practical implementations. The project's research agenda has foundations in classical optimization and more recent developments in optimization, control, learning theory, and statistics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.