权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Collaborative Research: Langevin Markov Chain Monte Carlo Methods for Machine Learning

合作研究：用于机器学习的朗之万马尔可夫链蒙特卡罗方法

基本信息

批准号：
2053485
负责人：
Mert Gurbuzbalaban
金额：
$ 18万
依托单位：
Rutgers University Newark
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-06-01 至 2024-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2053485&HistoricalAwards=false
关键词：
Collaborative Research Langevin Markov Chain

项目摘要

The research in this project will focus on a particular class of algorithms for machine learning and data science. In particular, the investigators consider the large class of Markov Chain Monte Carlo (MCMC) methods which arise in several contexts in machine learning and data science. The project will develop new algorithms within the subclass called Langevin MCMC methods. These new algorithms will be scalable to high dimensions and large datasets and will be faster than traditional ones. The features of scalability and fast convergence are important for use in Bayesian statistical inference as well as in non-convex stochastic optimization methods for machine learning. The algorithms will allow efficient training and calibration of predictive machine learning models from large-scale data and have a direct impact on a broad range of data-driven application areas from information technology to computer vision. Graduate students will be trained and involved in research. In this project, the PIs investigate a new class of algorithms within the class of Langevin MCMC methods. These algorithms can be applied in three contexts of machine learning and data science. First, they can be used for Bayesian (learning) inference problems with high-dimensional models, where the objective is to sample from a posterior distribution given a prior distribution on the parameter space and the likelihood of the observed data. Second, they can be used for solving stochastic non-convex optimization problems including the challenging problems arising in deep learning. Third, they arise in modeling and approximating workhorse algorithms in data science such as stochastic gradient descent methods. By leveraging out the connections between stochastic gradient algorithms and MCMC algorithms, the proposed approach results in a new class of stochastic gradient algorithms called Hamiltonian Accelerated Stochastic Gradient that can outperform existing methods in deep learning practice. A first goal of the project is to study theoretical convergence properties of the proposed algorithms further to fill out the current gap between theory and practice, as well as to develop new scalable algorithms that can extend the existing framework. A second goal is to investigate existing Langevin algorithms further to provide non-asymptotic rigorous performance guarantees relevant to machine learning and data science practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的研究将专注于机器学习和数据科学的特定算法。特别是，研究人员考虑了在机器学习和数据科学中出现的大量马尔可夫链蒙特卡罗（MCMC）方法。该项目将在称为Langevin MCMC方法的子类中开发新算法。这些新算法将可扩展到高维和大型数据集，并且比传统算法更快。可扩展性和快速收敛的特性对于贝叶斯统计推断以及机器学习的非凸随机优化方法都很重要。这些算法将允许从大规模数据中有效训练和校准预测机器学习模型，并对从信息技术到计算机视觉的广泛数据驱动应用领域产生直接影响。研究生将接受培训并参与研究。在这个项目中，PI研究了Langevin MCMC方法中的一类新算法。这些算法可以应用于机器学习和数据科学的三种背景下。首先，它们可以用于高维模型的贝叶斯（学习）推理问题，其目标是从参数空间上的先验分布和观察数据的可能性的后验分布中进行采样。其次，它们可用于解决随机非凸优化问题，包括深度学习中出现的挑战性问题。第三，它们出现在数据科学中的建模和近似算法中，如随机梯度下降方法。通过利用随机梯度算法和MCMC算法之间的联系，该方法产生了一类新的随机梯度算法，称为Hamiltonian Accelerated Stochastic Gradient，可以在深度学习实践中优于现有方法。该项目的第一个目标是进一步研究所提出的算法的理论收敛特性，以填补目前理论与实践之间的差距，以及开发新的可扩展算法，可以扩展现有的框架。第二个目标是进一步研究现有的Langevin算法，为机器学习和数据科学实践提供非渐近的严格性能保证。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（17）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Differentially Private Accelerated Optimization Algorithms

DOI：
10.1137/20m1355847
发表时间：
2020-08
期刊：
SIAM J. Optim.
影响因子：
0
作者：
Nurdan Kuru;cS. .Ilker Birbil;Mert Gurbuzbalaban;S. Yıldırım
通讯作者：
Nurdan Kuru;cS. .Ilker Birbil;Mert Gurbuzbalaban;S. Yıldırım

L-DQN: An Asynchronous Limited-Memory Distributed Quasi-Newton Method

DOI：
10.1109/cdc45484.2021.9682985
发表时间：
2021-08
期刊：
2021 60th IEEE Conference on Decision and Control (CDC)
影响因子：
0
作者：
Bugra Can;Saeed Soori;M. Dehnavi;M. Gürbüzbalaban
通讯作者：
Bugra Can;Saeed Soori;M. Dehnavi;M. Gürbüzbalaban

A Stochastic Subgradient Method for Distributionally Robust Non-convex and Non-smooth Learning

DOI：
10.1007/s10957-022-02063-6
发表时间：
2022-07
期刊：
Journal of Optimization Theory and Applications
影响因子：
1.9
作者：
M. Gürbüzbalaban;A. Ruszczynski;Landi Zhu
通讯作者：
M. Gürbüzbalaban;A. Ruszczynski;Landi Zhu

Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections

DOI：
发表时间：
2021-02
期刊：
ArXiv
影响因子：
0
作者：
A. Camuto;Xiaoyu Wang-;Lingjiong Zhu;Chris C. Holmes;M. Gürbüzbalaban;Umut Simsekli
通讯作者：
A. Camuto;Xiaoyu Wang-;Lingjiong Zhu;Chris C. Holmes;M. Gürbüzbalaban;Umut Simsekli

Boundary Conditions for Linear Exit Time Gradient Trajectories Around Saddle Points: Analysis and Algorithm