CAREER: Advancing Constrained and Non-Convex Learning

机器学习已经成为解决许多决策问题的不可或缺的工具，例如自动驾驶。随着机器学习算法在决策中的应用日益广泛和多样化，对安全性、公平性、可解释性和泛化的要求也越来越高。这些新出现的问题给存在大数据和复杂数据的机器学习算法的设计带来了巨大的挑战。通过最小化无约束或简单约束凸目标的传统机器学习方法已变得越来越不能令人满意。该项目旨在通过设计和分析高效和有效的优化算法来解决新的机器学习范例中的计算挑战，从而促进具有复杂目标和约束的学习。该项目将增强从更多样化和更广泛的应用解决大规模、现实世界问题的能力。此外，该项目将努力传播机器学习和优化的意义，并为不同层次的学生提供良好的研究体验。尽管文献中已经研究了约束优化和非凸优化在机器学习中的应用，但仍然存在巨大的挑战和许多问题尚未解决。本项目的主要目的是设计和分析一套高效的优化算法和统计学习方法，以促进大规模复杂目标和约束下的机器学习。该项目的技术目标分为两个方面。第一个重点是(I)针对复杂的非凸目标，开发更快且可证明的随机学习算法；(Ii)通过设计高效的优化算法，利用先进的正则化和压缩方法来提高深度学习的泛化性能。第二个重点是(I)为复杂和复杂约束的学习设计计算高效的约束优化算法，以及(Ii)研究它们在对抗性学习、公平学习、可解释学习等方面的应用。开发的优化工具和技术将使机器学习中更先进的正则化和损失最小化方法成为可能，并将极大地影响其他领域，如运筹学、信号处理、数据挖掘等。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning has emerged to be an indispensable tool for addressing many decision-making problems, e.g., autonomous driving. As applications of machine learning algorithms for decision-making broaden and diversify, the requirements on security, fairness, interpretability and generalization have been pushed to higher standards. These emerging issues have brought great challenges to the design of machine learning algorithms in the presence of big and complex data. Traditional machine learning methods by minimizing an unconstrained or simply constrained convex objective have become increasingly unsatisfactory. This project seeks to advance learning with complex objectives and constraints by designing and analyzing efficient and effective optimization algorithms for addressing computational challenges in new machine learning paradigms. The project will enhance the ability to solve large-scale, real-world problems from more diverse and broad applications. Furthermore, the project will strive to communicate the significance of machine learning and optimization and provide excellent research experience to students at different levels.Although both constrained optimization and non-convex optimization have been studied and applied to machine learning in the literature, great challenges and many problems remain unaddressed. The primary focus of this project is to design and analyze a set of efficient optimization algorithms and statistical learning methods for advancing machine learning with complex objectives and constraints at large scale. The technical aims of the project are divided into two thrusts. The first thrust is to (i) develop faster and provable stochastic algorithms for learning with complicated non-convex objectives, and (ii) improve the generalization performance of deep learning by advanced regularization and compression methods through design of efficient optimization algorithms. The second thrust is to (i) design computationally efficient constrained optimization algorithms for learning with complicated and complex constraints, and (ii) investigate their applications in adversarial learning, fair learning, interpretable learning, etc. The optimization tools and techniques developed will enable more advanced regularization and loss minimization methods in machine learning, and should greatly influence other areas, such as operations research, signal processing, data mining, etc.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.