CAREER: Stein Variational Gradient Descent: A New Foundation for Inference

职业生涯:斯坦因变分梯度下降:新的推理基础

基本信息

  • 批准号:
    1846421
  • 负责人:
  • 金额:
    $ 49.98万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Continuing Grant
  • 财政年份:
    2019
  • 资助国家:
    美国
  • 起止时间:
    2019-09-01 至 2024-08-31
  • 项目状态:
    已结题

项目摘要

In recent years, machine learning (ML) and artificial intelligence (AI) have achieved remarkable progress. Smart computer programs can now categorize images better than humans, beat the world champion at Go, and make intelligent recommendations in areas from health care to education. Under the hood, many of these technologies are made possible by the idea of using highly flexible and structured probabilistic models to express and reason with complex phenomena. Meanwhile, probabilistic models required for modern machine learning systems are becoming increasingly complex, and the ability to compute probabilities efficiently becomes one of the main bottlenecks of modern learning systems. The goal of this project is to develop a new theoretical and algorithmic framework of efficient and approximate computation of probabilities for highly complex probabilistic models. This project provides research opportunities for undergraduates and it also develops educational modules for outreach activities to high school students and undergraduates.Markov chain Monte Carlo (MCMC) and variational inference (VI) have been the two major types of approximate inference algorithms that dominate the literature. However, both of them have their own critical weaknesses. MCMC is accurate but suffers from slow convergence; VI is typically faster but introduces deterministic errors and lacks theoretical guarantees. This project aims to introduce a new Stein variational paradigm for approximate inference that integrates the advantages of MCMC and VI, enabling algorithms that are as flexible and accurate as MCMC and as fast as VI. The key idea is to directly optimize a non-parametric particle-based representation to fit intractable distributions with fast deterministic gradient-based updates, which is made possible by integrating and generalizing key mathematical tools from Stein's method, optimal transport and interacting particle systems. A basic algorithm derived from this framework, called Stein variational gradient descent (SVGD), has already been found to be a powerful tool in a range of applications. This project extends this initial success to a higher level, by (1) systematically investigating basic theoretical problems, (2) developing more efficient and practical algorithms and software, and (3) demonstrating its power in various interdisciplinary applications, including reinforcement learning and molecule dynamics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
近年来,机器学习(ML)和人工智能(AI)取得了令人瞩目的进展。智能计算机程序现在可以比人类更好地对图像进行分类,击败世界围棋冠军,并在从医疗保健到教育等领域提出智能建议。在底层,许多这些技术都是通过使用高度灵活和结构化的概率模型来表达和推理复杂现象的想法而成为可能的。与此同时,现代机器学习系统所需的概率模型越来越复杂,有效计算概率的能力成为现代学习系统的主要瓶颈之一。该项目的目标是为高度复杂的概率模型开发一种新的有效和近似计算概率的理论和算法框架。该项目为本科生提供了研究机会,并开发了面向高中生和本科生的拓展活动的教育模块。马尔可夫链蒙特卡罗算法(MCMC)和变分推理算法(VI)是目前文献中主要的两种近似推理算法。然而,它们都有自己的关键弱点。MCMC精度高,但收敛速度慢;VI通常更快,但会引入确定性错误,并且缺乏理论保证。该项目旨在引入一种新的Stein变分范式,用于近似推理,该范式集成了MCMC和VI的优点,使算法像MCMC一样灵活和准确,像VI一样快速。关键思想是直接优化基于非参数粒子的表示,以适应具有快速确定性梯度更新的难处理分布,这是通过整合和推广Stein方法中的关键数学工具实现的。最佳输运和相互作用粒子系统。从这个框架衍生出来的一个基本算法,称为Stein变分梯度下降(SVGD),已经被发现是一个强大的工具,在一系列应用中。该项目通过(1)系统地研究基础理论问题,(2)开发更有效和实用的算法和软件,以及(3)展示其在各种跨学科应用中的能力,包括强化学习和分子动力学,将这一初步成功扩展到更高的水平。该奖项反映了美国国家科学基金会的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(23)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
  • DOI:
    10.18653/v1/2020.acl-main.317
  • 发表时间:
    2020-05
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Mao Ye;Chengyue Gong;Qiang Liu
  • 通讯作者:
    Mao Ye;Chengyue Gong;Qiang Liu
A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family
  • DOI:
  • 发表时间:
    2019-10
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Trambak Banerjee;Qiang Liu;Gourab Mukherjee;Wengunag Sun
  • 通讯作者:
    Trambak Banerjee;Qiang Liu;Gourab Mukherjee;Wengunag Sun
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
  • DOI:
    10.48550/arxiv.2209.03003
  • 发表时间:
    2022-09
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Xingchao Liu;Chengyue Gong;Qiang Liu
  • 通讯作者:
    Xingchao Liu;Chengyue Gong;Qiang Liu
Stein Self-Repulsive Dynamics: Benefits From Past Samples
斯坦因自排斥动力学:过去样本的好处
Nonlinear Stein Variational Gradient Descent for Learning Diversified Mixture Models
  • DOI:
  • 发表时间:
    2019-05
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Dilin Wang;Qiang Liu
  • 通讯作者:
    Dilin Wang;Qiang Liu
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Qiang Liu其他文献

Effect of Cutting Parameters on Minimum Quantity Lubrication Machining of High Strength Steel
切削参数对高强度钢微量润滑加工的影响
  • DOI:
    10.4028/www.scientific.net/msf.626-627.387
  • 发表时间:
    2009
  • 期刊:
  • 影响因子:
    0
  • 作者:
    L. Yan;S. Yuan;Qiang Liu
  • 通讯作者:
    Qiang Liu
Fluorescence enhancement of gold nanoclusters via Zn doping for biomedical applications
通过锌掺杂增强金纳米团簇的荧光用于生物医学应用
  • DOI:
    10.1039/c7ra13072d
  • 发表时间:
    2018-02
  • 期刊:
  • 影响因子:
    3.9
  • 作者:
    Yanqing Qiao;Ying Liu;Haixia Liu;Yonghui Li;Wei Long;Junying Wang;Xiaoyu Mu;Jing Chen;Haile Liu;Xueting Bai;Lingfang Liu;Yuan-Ming Sun;Qiang Liu;Meili Guo;Xiao-Dong Zhang
  • 通讯作者:
    Xiao-Dong Zhang
A Kind of Broadband Polarization Filter Based on Photonic Crystal Fiber with Nanoscale Gold Film
一种基于纳米金膜光子晶体光纤的宽带偏振滤光片
  • DOI:
    10.1007/s11468-016-0274-6
  • 发表时间:
    2017
  • 期刊:
  • 影响因子:
    3
  • 作者:
    Guangyao Wang;Shuguang Li;Xinyu Wang;Yunyan Zhao;Qiang Liu;Jianchen Zi;Hui Li;Hailiang Chen
  • 通讯作者:
    Hailiang Chen
Compact broadband circularly-polarized directional universal GNSS antenna with symmetric radiation pattern and stable near-zenith coverage
紧凑型宽带圆极化定向通用 GNSS 天线,具有对称辐射方向图和稳定的近天顶覆盖
Low-Cost, High-Strength Cellulose-based Quasi-Solid Polymer Electrolyte for Solid-State Lithium-Metal Batteries.
用于固态锂金属电池的低成本、高强度纤维素基准固态聚合物电解质。
  • DOI:
    10.1002/anie.202302767
  • 发表时间:
    2023
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Dai;H. Xiea;Qiang Liu;Kexin Mu;Zhennuo Song;Weijian Xu;Lei Tian;Caizhen Zhu;Jian Xu
  • 通讯作者:
    Jian Xu

Qiang Liu的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Qiang Liu', 18)}}的其他基金

CC* Integration-Large: Husker-Net: Open Nebraska End-to-End Wireless Edge Networks
CC* 大型集成:Husker-Net:开放内布拉斯加州端到端无线边缘网络
  • 批准号:
    2321699
  • 财政年份:
    2023
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Standard Grant
CRII: RI: Anytime Inference with Confidence Bounds for Graphical Models
CRII:RI:图形模型的随时推理与置信界限
  • 批准号:
    1830161
  • 财政年份:
    2018
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Standard Grant
CRII: RI: Anytime Inference with Confidence Bounds for Graphical Models
CRII:RI:图形模型的随时推理与置信界限
  • 批准号:
    1565796
  • 财政年份:
    2016
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Standard Grant
Common Application Platform for Low Voltage Network Management
低压网络管理通用应用平台
  • 批准号:
    EP/N508421/1
  • 财政年份:
    2015
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Research Grant

相似国自然基金

Bochner-Riesz平均及Stein球面平均相关问题研究
  • 批准号:
  • 批准年份:
    2021
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目
三维流形上的切触结构及其Stein填充
  • 批准号:
    11871332
  • 批准年份:
    2018
  • 资助金额:
    53.0 万元
  • 项目类别:
    面上项目
基于Stein方法与Malliavin分析的自相似过程渐近行为研究及相关问题
  • 批准号:
    11401313
  • 批准年份:
    2014
  • 资助金额:
    22.0 万元
  • 项目类别:
    青年科学基金项目
Stein方法和几类相依随机变量的强极限定理
  • 批准号:
    11101364
  • 批准年份:
    2011
  • 资助金额:
    18.0 万元
  • 项目类别:
    青年科学基金项目

相似海外基金

Harnessing the Power of Stein Discrepancies in Bayesian Computation
利用贝叶斯计算中斯坦因差异的力量
  • 批准号:
    EP/W019590/1
  • 财政年份:
    2022
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Research Grant
Modernist whimsy: entertaining difficulty in the work of Marianne Moore, Gertrude Stein, and Stevie Smith
现代主义奇思妙想:玛丽安·摩尔、格特鲁德·斯坦和史蒂维·史密斯作品中的娱乐难题
  • 批准号:
    2773616
  • 财政年份:
    2022
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Studentship
An approach to Stein phenomenon from the viewpoint of thermodynamics
从热力学角度探讨斯坦因现象
  • 批准号:
    20K11711
  • 财政年份:
    2020
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
High-Dimensional Bayesian Inference with Stein Points
使用 Stein 点进行高维贝叶斯推理
  • 批准号:
    20K19763
  • 财政年份:
    2020
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
Flexible Stein Manifolds and Fukaya Categories
灵活的 Stein 流形和 Fukaya 类别
  • 批准号:
    1906564
  • 财政年份:
    2019
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Continuing Grant
Modernism and Autobiography: Stein, Hemingway and Fitzgerald
现代主义与自传:斯坦因、海明威和菲茨杰拉德
  • 批准号:
    18K00425
  • 财政年份:
    2018
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Whether the self-government state is contradictory. Study on theory of self-government by Lorenz von Stein
自治国家是否矛盾。
  • 批准号:
    22520079
  • 财政年份:
    2010
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Symplectic homology and Stein manifolds
辛同调和斯坦因流形
  • 批准号:
    1005365
  • 财政年份:
    2010
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Standard Grant
Die Werkstätten der Streitwagengarnison von Quantir/Pi-Ramesse (Grabungsplatz Q1). Stein- und Metallwerkzeuge
Quantir/Pi-Ramesse 战车驻军的作坊(发掘地点 Q1)。
  • 批准号:
    192606818
  • 财政年份:
    2010
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Publication Grants
CDA - Investigating socio-economic transformations in highland Southwest Iran during the 5th millennium BC: the Stein Collection in the British Museum
CDA - 调查公元前 5 世纪伊朗西南部高地的社会经济转型:大英博物馆的斯坦因收藏
  • 批准号:
    AH/I506624/1
  • 财政年份:
    2010
  • 资助金额:
    $ 49.98万
  • 项目类别:
    Training Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了