Energetic Variational Inference: Foundations, Algorithms, and Applications

能量变分推理：基础、算法和应用

• 批准号: 2153029
• 负责人: Lulu Kang
• 金额: $ 30万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153029&HistoricalAwards=false
• 关键词: Energetic; Variational; Inference; Foundations; Algorithms

Variational Inference is a powerful tool used to boost efficiency and flexibility in machine learning and artificial intelligence algorithms, particularly those based on large amounts of data. In this project, the investigators plan to create a unified and systematic framework for variational inference methods, making two key contributions. First, the investigators will establish the theoretical foundations for the proposed framework, which will support and justify using existing and new variational inference algorithms in machine learning applications. Second, the investigators will provide a systemic procedure to create new variational inference algorithms and apply them to emerging machine learning problems. In addition to these new scientific developments, the investigators will create new courses and workshops on machine learning, recruit both undergraduate and graduate students for summer, project-based research programs, and provide mentorship to local high school students through hands-on machine learning training programs. Collaborations are planned with industrial data science partners to apply these new algorithms in practice and to train the workforce with the start-of-the-art machine learning tools.The proposed "Energetic Variational Inference" framework is based on an energetic variational approach, which has been successfully used to study complicated non-equilibrium systems in physics and biology. The investigators will provide a blueprint for generating new algorithms by introducing various options for the four essential components of the proposed framework: the divergence functional, the dissipation functional, the representation of the probability density, and the temporal discretization. The investigators will study convergence in the continuous formulation as well as estimate the error bounds after temporal discretization of the underlying continuous dynamic system. More importantly, these theoretical results can be applied or extended to other flow-based variational inference approaches. These methods will be applied to problems in supervised learning, density estimation, and generative learning. Additional novel applications in machine learning, statistics, and statistical physics will also be developed. The algorithms will be packaged into open-source software for public use.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

变分推理是一个强大的工具，用于提高机器学习和人工智能算法的效率和灵活性，特别是那些基于大量数据的算法。在这个项目中，研究人员计划为变分推理方法创建一个统一和系统的框架，做出两个关键贡献。首先，研究人员将为所提出的框架建立理论基础，这将支持和证明在机器学习应用中使用现有和新的变分推理算法。其次，研究人员将提供一个系统的程序来创建新的变分推理算法，并将其应用于新兴的机器学习问题。除了这些新的科学发展，研究人员还将创建新的机器学习课程和研讨会，招募本科生和研究生参加暑期项目研究计划，并通过动手机器学习培训计划为当地高中生提供指导。计划与工业数据科学合作伙伴合作，将这些新算法应用于实践，并使用最先进的机器学习工具培训员工。拟议的“能量变分推理”框架基于能量变分方法，该方法已成功用于研究物理学和生物学中的复杂非平衡系统。 研究人员将通过为所提出的框架的四个基本组成部分引入各种选项来提供生成新算法的蓝图：发散泛函，耗散泛函，概率密度的表示和时间离散化。研究人员将研究连续公式的收敛性，并估计底层连续动态系统时间离散化后的误差界。更重要的是，这些理论结果可以应用或扩展到其他基于流的变分推理方法。这些方法将应用于监督学习，密度估计和生成学习中的问题。还将开发机器学习，统计学和统计物理学中的其他新应用。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Bayesian D-Optimal Design of Experiments with Quantitative and Qualitative Responses

具有定量和定性响应的贝叶斯 D 优化实验设计

• DOI: 10.51387/23-nejsds30
• 发表时间: 2023
• 期刊: The New England Journal of Statistics in Data Science
• 作者: Kang, Lulu;Deng, Xinwei;Jin, Ran
• 通讯作者: Jin, Ran