Energetic Variational Inference: Foundations, Algorithms, and Applications

变分推理是一种强大的工具，用于提高机器学习和人工智能算法的效率和灵活性，特别是那些基于大量数据的算法。在这个项目中，研究人员计划为变分推理方法创建一个统一和系统的框架，做出两个关键贡献。首先，研究人员将为拟议的框架建立理论基础，该框架将支持并证明在机器学习应用中使用现有的和新的变分推理算法。其次，研究人员将提供一个系统的程序来创建新的变分推理算法，并将它们应用于新出现的机器学习问题。除了这些新的科学发展，调查人员还将开设关于机器学习的新课程和研讨会，招募本科生和研究生参加暑期项目和基于项目的研究项目，并通过实际操作的机器学习培训项目为当地高中生提供指导。计划与工业数据科学合作伙伴合作，将这些新算法应用于实践，并使用最先进的机器学习工具培训工作人员。拟议的“能量变分推理”框架基于能量变分方法，已成功用于研究物理和生物学中的复杂非平衡系统。研究人员将通过为拟议框架的四个基本组件引入各种选项来提供生成新算法的蓝图：发散泛函、耗散泛函、概率密度表示和时间离散化。研究人员将研究连续公式中的收敛，并估计基本连续动力系统的时间离散化后的误差界。更重要的是，这些理论结果可以应用或推广到其他基于流的变分推理方法。这些方法将应用于监督学习、密度估计和产生式学习中的问题。还将开发机器学习、统计学和统计物理中的其他新应用程序。这些算法将被打包成开源软件供公众使用。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Variational Inference is a powerful tool used to boost efficiency and flexibility in machine learning and artificial intelligence algorithms, particularly those based on large amounts of data. In this project, the investigators plan to create a unified and systematic framework for variational inference methods, making two key contributions. First, the investigators will establish the theoretical foundations for the proposed framework, which will support and justify using existing and new variational inference algorithms in machine learning applications. Second, the investigators will provide a systemic procedure to create new variational inference algorithms and apply them to emerging machine learning problems. In addition to these new scientific developments, the investigators will create new courses and workshops on machine learning, recruit both undergraduate and graduate students for summer, project-based research programs, and provide mentorship to local high school students through hands-on machine learning training programs. Collaborations are planned with industrial data science partners to apply these new algorithms in practice and to train the workforce with the start-of-the-art machine learning tools.The proposed "Energetic Variational Inference" framework is based on an energetic variational approach, which has been successfully used to study complicated non-equilibrium systems in physics and biology. The investigators will provide a blueprint for generating new algorithms by introducing various options for the four essential components of the proposed framework: the divergence functional, the dissipation functional, the representation of the probability density, and the temporal discretization. The investigators will study convergence in the continuous formulation as well as estimate the error bounds after temporal discretization of the underlying continuous dynamic system. More importantly, these theoretical results can be applied or extended to other flow-based variational inference approaches. These methods will be applied to problems in supervised learning, density estimation, and generative learning. Additional novel applications in machine learning, statistics, and statistical physics will also be developed. The algorithms will be packaged into open-source software for public use.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.