CAREER: Nonconvex Optimization for Statistical Estimation and Learning: Conditioning, Dynamics, and Nonsmoothness

职业：统计估计和学习的非凸优化：条件、动力学和非平滑性

• 批准号: 2047637
• 负责人: Damek Davis
• 金额: $ 45.4万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-02-15 至 2026-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047637&HistoricalAwards=false
• 关键词: CAREER; Nonconvex; Optimization; Statistical; Estimation

Nonconvex statistical estimation and learning algorithms are dramatically improving our capacity to efficiently learn from massive datasets, reshaping society through new technological capabilities in healthcare, imaging, transportation, and information processing. Although such learning algorithms have had widespread empirical success, we have yet to find a coherent mathematical foundation that can explain not only why they work and what tasks they provably solve, but also how practitioners can improve their performance either by adjusting the algorithm or even the task itself. The investigator aims to lay this foundation by advancing the design, analysis, and deployment of rigorously justified nonconvex optimization algorithms. This research will create guaranteed procedures for training practical machine learning systems deployed in government and industry, producing more reliable and robust predictive models with fewer data and computational resources. The investigator will incorporate results from this project in education efforts, including course development, local K-12 outreach, and research mentoring of Ph.D. and undergraduate students.In this project, the investigator designs and analyzes nonconvex optimization algorithms. The project focuses on simple iterative methods that compute with data in its ambient form, a class of algorithms that are uniquely scalable to modern high-dimensional statistical estimation and learning tasks. The overarching goal of the project is to understand when these methods converge to local or global optima and to provide efficiency estimates of their performance, measured both in terms of data and computational resources consumed. To achieve this goal, the investigation will draw on the techniques of variational analysis, nonsmooth optimization, machine learning, statistics, and high-dimensional probability. The investigator will leverage these techniques to design and equip simple, scalable iterative methods for nonconvex data fitting problems with strong performance guarantees: generic initialization strategies, rapid local convergence near optima, and seamless adaptation to nonsmooth constraints, models, priors. Such performance guarantees guide the practical implementation of reliable and efficient numerical methods for high-dimensional estimation and learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

非凸统计估计和学习算法极大地提高了我们从海量数据集中有效学习的能力，通过医疗保健、成像、交通和信息处理方面的新技术能力重塑了社会。尽管这样的学习算法在经验上取得了广泛的成功，但我们还没有找到一个连贯的数学基础，不仅可以解释它们为什么工作，它们可以解决什么任务，而且还可以解释从业者如何通过调整算法甚至任务本身来提高他们的表现。研究者的目标是通过推进设计、分析和部署严格合理的非凸优化算法来奠定这一基础。这项研究将为训练部署在政府和工业中的实用机器学习系统创建有保障的程序，以更少的数据和计算资源产生更可靠、更强大的预测模型。研究者将把这个项目的成果纳入教育工作，包括课程开发，当地K-12的推广，以及博士和本科生的研究指导。在这个项目中，研究者设计并分析了非凸优化算法。该项目侧重于使用环境形式的数据进行计算的简单迭代方法，这是一类可独特扩展到现代高维统计估计和学习任务的算法。该项目的总体目标是了解这些方法何时收敛到局部或全局最优，并根据所消耗的数据和计算资源对其性能进行效率估计。为了实现这一目标，调查将利用变分分析、非光滑优化、机器学习、统计和高维概率等技术。研究者将利用这些技术来设计和装备简单的、可扩展的迭代方法，用于非凸数据拟合问题，并具有强大的性能保证：通用初始化策略，接近最优的快速局部收敛，以及对非光滑约束、模型和先验的无缝适应。这种性能保证指导了高维估计和学习的可靠和有效的数值方法的实际实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

A Superlinearly Convergent Subgradient Method for Sharp Semismooth Problems

• DOI: 10.1287/moor.2023.1390
• 发表时间: 2022-01
• 期刊: Mathematics of Operations Research
• 影响因子: 1.7
• 作者: Vasileios Charisopoulos;Damek Davis

ESCAPING STRICT SADDLE POINTS OF THE MOREAU ENVELOPE IN NONSMOOTH OPTIMIZATION

• DOI: 10.1137/21m1430868
• 发表时间: 2022-01-01
• 期刊: SIAM JOURNAL ON OPTIMIZATION
• 影响因子: 3.1
• 作者: Davis, Damek;Diaz, Mateo;Drusvyatskiy, Dmitriy
• 通讯作者: Drusvyatskiy, Dmitriy

Conservative and Semismooth Derivatives are Equivalent for Semialgebraic Maps

半代数映射的保守导数和半光滑导数是等价的

• DOI: 10.1007/s11228-021-00594-0
• 发表时间: 2021
• 期刊: Set-Valued and Variational Analysis
• 影响因子: 1.6
• 作者: Davis, Damek;Drusvyatskiy, Dmitriy
• 通讯作者: Drusvyatskiy, Dmitriy

A gradient sampling method with complexity guarantees for Lipschitz functions in high and low dimensions

• 发表时间: 2021-12
• 作者: Damek Davis;D. Drusvyatskiy;Y. Lee;Swati Padmanabhan;Guanghao Ye

Stochastic algorithms with geometric step decay converge linearly on sharp functions

• DOI: 10.1007/s10107-023-02003-w
• 发表时间: 2019-07
• 期刊: ArXiv
• 作者: Damek Davis;D. Drusvyatskiy;Vasileios Charisopoulos