CAREER: Advancing Constrained and Non-Convex Learning

职业：推进约束和非凸学习

• 批准号: 2246753
• 负责人: Tianbao Yang
• 金额: $ 52.91万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2024-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2246753&HistoricalAwards=false
• 关键词: CAREER; Advancing; Constrained; Non; Convex

Machine learning has emerged to be an indispensable tool for addressing many decision-making problems, e.g., autonomous driving. As applications of machine learning algorithms for decision-making broaden and diversify, the requirements on security, fairness, interpretability and generalization have been pushed to higher standards. These emerging issues have brought great challenges to the design of machine learning algorithms in the presence of big and complex data. Traditional machine learning methods by minimizing an unconstrained or simply constrained convex objective have become increasingly unsatisfactory. This project seeks to advance learning with complex objectives and constraints by designing and analyzing efficient and effective optimization algorithms for addressing computational challenges in new machine learning paradigms. The project will enhance the ability to solve large-scale, real-world problems from more diverse and broad applications. Furthermore, the project will strive to communicate the significance of machine learning and optimization and provide excellent research experience to students at different levels.Although both constrained optimization and non-convex optimization have been studied and applied to machine learning in the literature, great challenges and many problems remain unaddressed. The primary focus of this project is to design and analyze a set of efficient optimization algorithms and statistical learning methods for advancing machine learning with complex objectives and constraints at large scale. The technical aims of the project are divided into two thrusts. The first thrust is to (i) develop faster and provable stochastic algorithms for learning with complicated non-convex objectives, and (ii) improve the generalization performance of deep learning by advanced regularization and compression methods through design of efficient optimization algorithms. The second thrust is to (i) design computationally efficient constrained optimization algorithms for learning with complicated and complex constraints, and (ii) investigate their applications in adversarial learning, fair learning, interpretable learning, etc. The optimization tools and techniques developed will enable more advanced regularization and loss minimization methods in machine learning, and should greatly influence other areas, such as operations research, signal processing, data mining, etc.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习已经成为解决许多决策问题的不可或缺的工具，例如，自动驾驶随着机器学习算法在决策中的应用越来越广泛和多样化，对安全性、公平性、可解释性和泛化性的要求也越来越高。这些新出现的问题给在大而复杂的数据中设计机器学习算法带来了巨大的挑战。 传统的机器学习方法通过最小化无约束或简单约束的凸目标变得越来越不令人满意。该项目旨在通过设计和分析高效和有效的优化算法来解决新机器学习范式中的计算挑战，从而推进具有复杂目标和约束的学习。该项目将提高从更多样化和更广泛的应用中解决大规模现实问题的能力。此外，本项目还将努力向不同层次的学生传达机器学习和优化的重要性，并提供优秀的研究经验。虽然在文献中，约束优化和非凸优化都已被研究并应用于机器学习，但仍存在巨大的挑战和许多问题尚未解决。该项目的主要重点是设计和分析一套有效的优化算法和统计学习方法，用于大规模推进具有复杂目标和约束的机器学习。该项目的技术目标分为两个方面。第一个目标是（i）开发更快和可证明的随机算法，用于复杂非凸目标的学习，以及（ii）通过设计高效的优化算法，通过高级正则化和压缩方法来提高深度学习的泛化性能。第二个目标是（i）设计计算效率高的约束优化算法，用于复杂和复杂约束的学习，以及（ii）研究它们在对抗学习，公平学习，可解释学习等方面的应用。开发的优化工具和技术将使机器学习中更先进的正则化和损失最小化方法成为可能，并将极大地影响其他领域，如运筹学，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization

• DOI: 10.48550/arxiv.2207.08540
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: Wei Jiang;Gang Li;Yibo Wang-;Lijun Zhang;Tianbao Yang

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

• 发表时间: 2021-06
• 期刊: J. Mach. Learn. Res.
• 作者: Bokun Wang;Zhuoning Yuan;Yiming Ying;Tianbao Yang

Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization

• DOI: 10.48550/arxiv.2206.00260
• 发表时间: 2022-06
• 期刊: ArXiv
• 作者: Quanqi Hu;Yongjian Zhong;Tianbao Yang

Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints

• DOI: 10.48550/arxiv.2212.12603
• 发表时间: 2022-12
• 作者: Yao Yao-Yao;Qihang Lin;Tianbao Yang

Large-scale Optimization of Partial AUC in a Range of False Positive Rates

• DOI: 10.48550/arxiv.2203.01505
• 发表时间: 2022-03
• 期刊: ArXiv
• 作者: Yao Yao-Yao;Qihang Lin;Tianbao Yang