权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Modern nonconvex optimization for machine learning: foundations of geometric and scalable techniques

职业：机器学习的现代非凸优化：几何和可扩展技术的基础

基本信息

批准号：
1846088
负责人：
Suvrit Sra
金额：
$ 50万
依托单位：
Massachusetts Institute of Technology
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-03-15 至 2024-02-29
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1846088&HistoricalAwards=false
关键词：
CAREER Modern nonconvex optimization machine

项目摘要

Mathematical optimization lies at the heart of machine learning (ML) and artificial intelligence (AI) algorithms. Key challenges herein are to decide what criteria to optimize, and what algorithms to use for performing the optimization. These challenges underlie the motivation for the present project. More specifically, this project seeks to make progress on three fundamental topics in optimization for ML: (i) theoretical foundations for a rich new class of optimization problems that can be solved efficiently (i.e., in a computationally tractable manner); (ii) a set of algorithms that apply to large-scale optimization problems in machine learning (e.g., for accelerating the training of neural networks); and (iii) theory that seeks to understand and explain why do neural networks succeed in practice. By focusing on topics of foundational importance, this project should spur a variety of followup research that deepends the connection of ML and AI with both mathematics and the applied sciences. More broadly, the this project may have a lasting societal impact too, primarily because of (i) its focus on optimization particularly relevant to ML and AI; (2) the non-traditional application domains it connects with (e.g., synthetic biology); and (3) because the investigator is in an environment that fosters such impact (namely, the Institute for Data, Systems, and Society (IDSS), a cross-disciplinary institute at MIT whose mission to drive solutions to problems of societal relevance). Finally, the project has an education centric focus; it involves intellectual and professional development of students, as well as development of curricular material based on the topics of research covered herein.This project lays out an ambitious agenda to develop foundational theory for geometric optimization, large-scale nonconvex optimization, and deep neural networks. The research on geometric optimization (which is a powerful new subclass of nonconvex optimization), is originally motivated by applications in ML and statistics; however, it stands to have a broader impact across all disciplines that consume optimization. The investigator seeks to develop a theory of polynomial time optimization for a class strictly larger than usual convex optimization problems, and thereby endow practitioners with new polynomial time tools and models; if successful, this investigation could open an entire subarea of research and applications. Beyond geometric optimization, the project also focuses on large-scale nonconvex optimization and on the theory of optimization and generalization for deep learning. Within these topics, the project will address key theoretical challenges, develop scalable new algorithms that could greatly speed up neural network training, and also make progress that reduces the gap between the theory and real-world practice of nonconvex optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数学优化是机器学习（ML）和人工智能（AI）算法的核心。这里的关键挑战是决定要优化的标准，以及使用什么算法来执行优化。这些挑战构成了当前项目的动机。更具体地说，该项目寻求在机器学习优化的三个基本主题上取得进展：(i)可以有效解决的丰富的新一类优化问题的理论基础（即，以计算可处理的方式）；（ii）一组适用于机器学习中大规模优化问题的算法（例如，用于加速神经网络的训练）；（iii）试图理解和解释为什么神经网络在实践中取得成功的理论。通过关注具有基础重要性的主题，该项目应该激发各种后续研究，深化ML和AI与数学和应用科学的联系。更广泛地说，这个项目也可能产生持久的社会影响，主要是因为(i)它专注于与ML和AI特别相关的优化；(2)与之相关的非传统应用领域（如合成生物学）；(3)因为研究者所在的环境能够促进这种影响（即数据、系统和社会研究所（IDSS），这是麻省理工学院的一个跨学科研究所，其使命是推动社会相关问题的解决方案）。最后，该项目以教育为中心；它涉及学生的智力和专业发展，以及基于本文所涵盖的研究主题的课程材料的开发。该项目制定了一个雄心勃勃的议程，以发展几何优化，大规模非凸优化和深度神经网络的基础理论。几何优化（非凸优化的一个强大的新子类）的研究最初是由ML和统计学中的应用驱动的；然而，它会对所有需要优化的学科产生更广泛的影响。研究者寻求发展一个多项式时间优化理论的类严格大于通常的凸优化问题，从而赋予从业者新的多项式时间工具和模型；如果成功，这项研究将打开一个完整的子领域的研究和应用。除了几何优化之外，该项目还专注于大规模非凸优化以及深度学习的优化和泛化理论。在这些主题中，该项目将解决关键的理论挑战，开发可扩展的新算法，这些算法可以大大加快神经网络的训练速度，并在缩小非凸优化理论与现实世界实践之间的差距方面取得进展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。