CAREER: Stein Variational Gradient Descent: A New Foundation for Inference
职业生涯:斯坦因变分梯度下降:新的推理基础
基本信息
- 批准号:1846421
- 负责人:
- 金额:$ 49.98万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2019
- 资助国家:美国
- 起止时间:2019-09-01 至 2024-08-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
In recent years, machine learning (ML) and artificial intelligence (AI) have achieved remarkable progress. Smart computer programs can now categorize images better than humans, beat the world champion at Go, and make intelligent recommendations in areas from health care to education. Under the hood, many of these technologies are made possible by the idea of using highly flexible and structured probabilistic models to express and reason with complex phenomena. Meanwhile, probabilistic models required for modern machine learning systems are becoming increasingly complex, and the ability to compute probabilities efficiently becomes one of the main bottlenecks of modern learning systems. The goal of this project is to develop a new theoretical and algorithmic framework of efficient and approximate computation of probabilities for highly complex probabilistic models. This project provides research opportunities for undergraduates and it also develops educational modules for outreach activities to high school students and undergraduates.Markov chain Monte Carlo (MCMC) and variational inference (VI) have been the two major types of approximate inference algorithms that dominate the literature. However, both of them have their own critical weaknesses. MCMC is accurate but suffers from slow convergence; VI is typically faster but introduces deterministic errors and lacks theoretical guarantees. This project aims to introduce a new Stein variational paradigm for approximate inference that integrates the advantages of MCMC and VI, enabling algorithms that are as flexible and accurate as MCMC and as fast as VI. The key idea is to directly optimize a non-parametric particle-based representation to fit intractable distributions with fast deterministic gradient-based updates, which is made possible by integrating and generalizing key mathematical tools from Stein's method, optimal transport and interacting particle systems. A basic algorithm derived from this framework, called Stein variational gradient descent (SVGD), has already been found to be a powerful tool in a range of applications. This project extends this initial success to a higher level, by (1) systematically investigating basic theoretical problems, (2) developing more efficient and practical algorithms and software, and (3) demonstrating its power in various interdisciplinary applications, including reinforcement learning and molecule dynamics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
近年来,机器学习(ML)和人工智能(AI)取得了令人瞩目的进展。智能计算机程序现在可以比人类更好地对图像进行分类,在围棋比赛中击败世界冠军,并在从医疗保健到教育等领域提出智能建议。在引擎盖下,这些技术中的许多技术都是通过使用高度灵活和结构化的概率模型来表达和推理复杂现象的想法而实现的。与此同时,现代机器学习系统所需的概率模型变得越来越复杂,有效计算概率的能力成为现代学习系统的主要瓶颈之一。该项目的目标是为高度复杂的概率模型开发一个新的理论和算法框架,用于有效和近似计算概率。该项目为本科生提供研究机会,并开发教育模块,为高中生和本科生的外展活动。马尔可夫链蒙特卡罗(MCMC)和变分推理(VI)一直是两种主要类型的近似推理算法,占主导地位的文献。然而,两者都有自己的致命弱点。MCMC是准确的,但收敛速度慢; VI通常更快,但引入了确定性错误,缺乏理论保证。 该项目旨在引入一种新的Stein变分范式进行近似推理,该范式集成了MCMC和VI的优点,使算法与MCMC一样灵活和准确,与VI一样快速。其关键思想是直接优化非参数的基于粒子的表示,以适应棘手的分布与快速确定性的基于梯度的更新,这是通过整合和推广关键的数学工具,从斯坦的方法,最佳运输和相互作用的粒子系统。一个基本的算法来自这个框架,称为斯坦变分梯度下降(SVGD),已经被发现是一个强大的工具,在一系列的应用。该项目将这一最初的成功扩展到更高的水平,通过(1)系统地研究基本理论问题,(2)开发更有效和实用的算法和软件,以及(3)在各种跨学科应用中展示其力量,包括强化学习和分子动力学。该奖项反映了NSF的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准。
项目成果
期刊论文数量(23)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
- DOI:10.18653/v1/2020.acl-main.317
- 发表时间:2020-05
- 期刊:
- 影响因子:0
- 作者:Mao Ye;Chengyue Gong;Qiang Liu
- 通讯作者:Mao Ye;Chengyue Gong;Qiang Liu
A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family
- DOI:
- 发表时间:2019-10
- 期刊:
- 影响因子:0
- 作者:Trambak Banerjee;Qiang Liu;Gourab Mukherjee;Wengunag Sun
- 通讯作者:Trambak Banerjee;Qiang Liu;Gourab Mukherjee;Wengunag Sun
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
- DOI:10.48550/arxiv.2209.03003
- 发表时间:2022-09
- 期刊:
- 影响因子:0
- 作者:Xingchao Liu;Chengyue Gong;Qiang Liu
- 通讯作者:Xingchao Liu;Chengyue Gong;Qiang Liu
Stein Self-Repulsive Dynamics: Benefits From Past Samples
斯坦因自排斥动力学:过去样本的好处
- DOI:
- 发表时间:2020
- 期刊:
- 影响因子:0
- 作者:Ye, Mao;Ren, Tongzheng;Liu, Qiang
- 通讯作者:Liu, Qiang
Nonlinear Stein Variational Gradient Descent for Learning Diversified Mixture Models
- DOI:
- 发表时间:2019-05
- 期刊:
- 影响因子:0
- 作者:Dilin Wang;Qiang Liu
- 通讯作者:Dilin Wang;Qiang Liu
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Qiang Liu其他文献
Effects of radio-frequency energy on peroxidase inactivation and physiochemical properties of stem lettuce and the underlying cell-morphology mechanism
射频能量对生菜过氧化物酶失活和理化特性的影响及其细胞形态机制
- DOI:
10.1016/j.foodchem.2020.126753 - 发表时间:
2020 - 期刊:
- 影响因子:8.8
- 作者:
Yishun Yao;Xunjiao Wei;Huiyun Pang;Ke Wang;Qiang Liu;Hongfei Fu;Xiangwei Chen;Yunyang Wang - 通讯作者:
Yunyang Wang
Rapid exploration of the epitope coverage produced by an Ebola survivor to guide the discovery of therapeutic antibody cocktails
快速探索埃博拉幸存者产生的表位覆盖范围,以指导治疗性抗体混合物的发现
- DOI:
- 发表时间:
2020 - 期刊:
- 影响因子:0
- 作者:
Tom Z. Yuan;Ana G. Lujan Hernandez;Erica Keane;Qiang Liu;F. Axelrod;Shweta Kailasan;Madeleine Noonan;M. Aman;A. Sato;Y. Abdiche - 通讯作者:
Y. Abdiche
Sequential hypothermic and normothermic perfusion preservation and transplantation of expanded criteria donor livers.
扩大标准供体肝脏的顺序低温和常温灌注保存和移植。
- DOI:
- 发表时间:
2022 - 期刊:
- 影响因子:3.8
- 作者:
Qiang Liu;Luca Del Prete;K. Ali;Patrick M. Grady;M. Bilancini;John Etterling;Giuseppe D’amico;T. Diago Uso;K. Hashimoto;F. Aucejo;M. Fujiki;B. Eghtesad;K. Sasaki;C. Kwon;S. Chaudhry;J. Doi;Alejandro Pita;Brandon New;A. Bennett;J. Cywinski;C. Miller;C. Quintini - 通讯作者:
C. Quintini
Eliminating the influence of axle parameters in bridge influence line identification based on long-gauge strain sensing
基于长应变传感的桥梁影响线识别中消除车轴参数的影响
- DOI:
- 发表时间:
2021 - 期刊:
- 影响因子:0
- 作者:
Qingqing Zhang;Qianlong Liu;Li Dai;Qiang Liu - 通讯作者:
Qiang Liu
Novel wideband single-layer 90° phase shifter based on radial-stub and coupled-line
基于径向短截线和耦合线的新型宽带单层90°移相器
- DOI:
- 发表时间:
2016 - 期刊:
- 影响因子:0
- 作者:
Qiang Liu;Yong Li;Yuanan Liu - 通讯作者:
Yuanan Liu
Qiang Liu的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Qiang Liu', 18)}}的其他基金
CC* Integration-Large: Husker-Net: Open Nebraska End-to-End Wireless Edge Networks
CC* 大型集成:Husker-Net:开放内布拉斯加州端到端无线边缘网络
- 批准号:
2321699 - 财政年份:2023
- 资助金额:
$ 49.98万 - 项目类别:
Standard Grant
CRII: RI: Anytime Inference with Confidence Bounds for Graphical Models
CRII:RI:图形模型的随时推理与置信界限
- 批准号:
1830161 - 财政年份:2018
- 资助金额:
$ 49.98万 - 项目类别:
Standard Grant
CRII: RI: Anytime Inference with Confidence Bounds for Graphical Models
CRII:RI:图形模型的随时推理与置信界限
- 批准号:
1565796 - 财政年份:2016
- 资助金额:
$ 49.98万 - 项目类别:
Standard Grant
Common Application Platform for Low Voltage Network Management
低压网络管理通用应用平台
- 批准号:
EP/N508421/1 - 财政年份:2015
- 资助金额:
$ 49.98万 - 项目类别:
Research Grant
相似国自然基金
Bochner-Riesz平均及Stein球面平均相关问题研究
- 批准号:
- 批准年份:2021
- 资助金额:30 万元
- 项目类别:青年科学基金项目
三维流形上的切触结构及其Stein填充
- 批准号:11871332
- 批准年份:2018
- 资助金额:53.0 万元
- 项目类别:面上项目
基于Stein方法与Malliavin分析的自相似过程渐近行为研究及相关问题
- 批准号:11401313
- 批准年份:2014
- 资助金额:22.0 万元
- 项目类别:青年科学基金项目
Stein方法和几类相依随机变量的强极限定理
- 批准号:11101364
- 批准年份:2011
- 资助金额:18.0 万元
- 项目类别:青年科学基金项目
相似海外基金
Modernist whimsy: entertaining difficulty in the work of Marianne Moore, Gertrude Stein, and Stevie Smith
现代主义奇思妙想:玛丽安·摩尔、格特鲁德·斯坦和史蒂维·史密斯作品中的娱乐难题
- 批准号:
2773616 - 财政年份:2022
- 资助金额:
$ 49.98万 - 项目类别:
Studentship
Harnessing the Power of Stein Discrepancies in Bayesian Computation
利用贝叶斯计算中斯坦因差异的力量
- 批准号:
EP/W019590/1 - 财政年份:2022
- 资助金额:
$ 49.98万 - 项目类别:
Research Grant
An approach to Stein phenomenon from the viewpoint of thermodynamics
从热力学角度探讨斯坦因现象
- 批准号:
20K11711 - 财政年份:2020
- 资助金额:
$ 49.98万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
High-Dimensional Bayesian Inference with Stein Points
使用 Stein 点进行高维贝叶斯推理
- 批准号:
20K19763 - 财政年份:2020
- 资助金额:
$ 49.98万 - 项目类别:
Grant-in-Aid for Early-Career Scientists
Flexible Stein Manifolds and Fukaya Categories
灵活的 Stein 流形和 Fukaya 类别
- 批准号:
1906564 - 财政年份:2019
- 资助金额:
$ 49.98万 - 项目类别:
Continuing Grant
Modernism and Autobiography: Stein, Hemingway and Fitzgerald
现代主义与自传:斯坦因、海明威和菲茨杰拉德
- 批准号:
18K00425 - 财政年份:2018
- 资助金额:
$ 49.98万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
Whether the self-government state is contradictory. Study on theory of self-government by Lorenz von Stein
自治国家是否矛盾。
- 批准号:
22520079 - 财政年份:2010
- 资助金额:
$ 49.98万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
Symplectic homology and Stein manifolds
辛同调和斯坦因流形
- 批准号:
1005365 - 财政年份:2010
- 资助金额:
$ 49.98万 - 项目类别:
Standard Grant
Die Werkstätten der Streitwagengarnison von Quantir/Pi-Ramesse (Grabungsplatz Q1). Stein- und Metallwerkzeuge
Quantir/Pi-Ramesse 战车驻军的作坊(发掘地点 Q1)。
- 批准号:
192606818 - 财政年份:2010
- 资助金额:
$ 49.98万 - 项目类别:
Publication Grants
CDA - Investigating socio-economic transformations in highland Southwest Iran during the 5th millennium BC: the Stein Collection in the British Museum
CDA - 调查公元前 5 世纪伊朗西南部高地的社会经济转型:大英博物馆的斯坦因收藏
- 批准号:
AH/I506624/1 - 财政年份:2010
- 资助金额:
$ 49.98万 - 项目类别:
Training Grant