权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Collaborative Research: Langevin Markov Chain Monte Carlo Methods for Machine Learning

合作研究：用于机器学习的朗之万马尔可夫链蒙特卡罗方法

基本信息

批准号：
2053454
负责人：
Lingjiong Zhu
金额：
$ 16万
依托单位：
Florida State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-06-01 至 2025-05-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2053454&HistoricalAwards=false
关键词：
Collaborative Research Langevin Markov Chain

项目摘要

The research in this project will focus on a particular class of algorithms for machine learning and data science. In particular, the investigators consider the large class of Markov Chain Monte Carlo (MCMC) methods which arise in several contexts in machine learning and data science. The project will develop new algorithms within the subclass called Langevin MCMC methods. These new algorithms will be scalable to high dimensions and large datasets and will be faster than traditional ones. The features of scalability and fast convergence are important for use in Bayesian statistical inference as well as in non-convex stochastic optimization methods for machine learning. The algorithms will allow efficient training and calibration of predictive machine learning models from large-scale data and have a direct impact on a broad range of data-driven application areas from information technology to computer vision. Graduate students will be trained and involved in research. In this project, the PIs investigate a new class of algorithms within the class of Langevin MCMC methods. These algorithms can be applied in three contexts of machine learning and data science. First, they can be used for Bayesian (learning) inference problems with high-dimensional models, where the objective is to sample from a posterior distribution given a prior distribution on the parameter space and the likelihood of the observed data. Second, they can be used for solving stochastic non-convex optimization problems including the challenging problems arising in deep learning. Third, they arise in modeling and approximating workhorse algorithms in data science such as stochastic gradient descent methods. By leveraging out the connections between stochastic gradient algorithms and MCMC algorithms, the proposed approach results in a new class of stochastic gradient algorithms called Hamiltonian Accelerated Stochastic Gradient that can outperform existing methods in deep learning practice. A first goal of the project is to study theoretical convergence properties of the proposed algorithms further to fill out the current gap between theory and practice, as well as to develop new scalable algorithms that can extend the existing framework. A second goal is to investigate existing Langevin algorithms further to provide non-asymptotic rigorous performance guarantees relevant to machine learning and data science practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的研究将专注于机器学习和数据科学的特定算法。特别是，研究人员考虑了在机器学习和数据科学中出现的大量马尔可夫链蒙特卡罗（MCMC）方法。该项目将在称为Langevin MCMC方法的子类中开发新算法。这些新算法将可扩展到高维和大型数据集，并且比传统算法更快。可扩展性和快速收敛的特性对于贝叶斯统计推断以及机器学习的非凸随机优化方法都很重要。这些算法将允许从大规模数据中有效训练和校准预测机器学习模型，并对从信息技术到计算机视觉的广泛数据驱动应用领域产生直接影响。研究生将接受培训并参与研究。在这个项目中，PI研究了Langevin MCMC方法中的一类新算法。这些算法可以应用于机器学习和数据科学的三种背景下。首先，它们可以用于高维模型的贝叶斯（学习）推理问题，其目标是从参数空间上的先验分布和观察数据的可能性的后验分布中进行采样。其次，它们可用于解决随机非凸优化问题，包括深度学习中出现的挑战性问题。第三，它们出现在数据科学中的建模和近似算法中，如随机梯度下降方法。通过利用随机梯度算法和MCMC算法之间的联系，该方法产生了一类新的随机梯度算法，称为Hamiltonian Accelerated Stochastic Gradient，可以在深度学习实践中优于现有方法。该项目的第一个目标是进一步研究所提出的算法的理论收敛特性，以填补目前理论与实践之间的差距，以及开发新的可扩展算法，可以扩展现有的框架。第二个目标是进一步研究现有的Langevin算法，为机器学习和数据科学实践提供非渐近的严格性能保证。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（10）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Approximate Variational Estimation for a Model of Network Formation

DOI：
10.1162/rest_a_01023
发表时间：
2017-02
期刊：
Review of Economics and Statistics
影响因子：
8
作者：
A. Mele;Lingjiong Zhu
通讯作者：
A. Mele;Lingjiong Zhu

Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration

DOI：
10.1287/opre.2021.2162
发表时间：
2018-09
期刊：
Oper. Res.
影响因子：
0
作者：
Xuefeng Gao;M. Gürbüzbalaban;Lingjiong Zhu
通讯作者：
Xuefeng Gao;M. Gürbüzbalaban;Lingjiong Zhu

Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares

DOI：
10.48550/arxiv.2206.01274
发表时间：
2022-06
期刊：
影响因子：
0
作者：
Anant Raj;Melih Barsbey;M. Gürbüzbalaban;Lingjiong Zhu;Umut Simsekli
通讯作者：
Anant Raj;Melih Barsbey;M. Gürbüzbalaban;Lingjiong Zhu;Umut Simsekli

Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms

DOI：
发表时间：
2021-06
期刊：
影响因子：
0
作者：
A. Camuto;George Deligiannidis;Murat A. Erdogdu;Mert Gurbuzbalaban;Umut cSimcsekli;Lingjiong Zhu
通讯作者：
A. Camuto;George Deligiannidis;Murat A. Erdogdu;Mert Gurbuzbalaban;Umut cSimcsekli;Lingjiong Zhu

Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks