权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

III: Small: Algorithms and Theoretical Foundations for Approximate Bayesian Inference in Machine Learning

III：小：机器学习中近似贝叶斯推理的算法和理论基础

基本信息

批准号：
1714440
负责人：
Roni Khardon
金额：
$ 49.71万
依托单位：
Tufts University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-08-01 至 2019-01-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1714440&HistoricalAwards=false
关键词：
III Small Algorithms Theoretical Foundations

项目摘要

Over the last two decades Bayesian models have become central in machine learning. Bayesian models often hypothesize latent (non-observed) variables with explanatory or predictive power toward observed phenomena. The challenge is to infer the state of these variables or a belief over that state from observed data. For example, one might try to infer a user's preferences from observations about their own behavior and the behavior of other users. The goal of this project is to develop general approximate inference algorithms that work across large families of Bayesian models so that solutions can be widely reused. The algorithmic work will be complemented by developing a learning theory for approximate Bayesian inference in machine learning. The theoretical framework will aim to prove performance guarantees for Bayesian prediction algorithms and inform the design of algorithms with desirable properties. The project will contribute to basic scientific research, advancing core goals in machine learning. The project will support training and research of PhD students and therefore will directly support human development. Through classroom teaching and outreach the project will expose a larger population of students to machine learning and its potential in applications.More concretely, the project will investigate non-conjugate Bayesian latent variable models, i.e., it will avoid the often used but limiting simplifying assumption of conjugacy. On the algorithmic side the project will aim to generalize the paradigm of variational message passing for non-conjugate graphical models, and to develop stochastic variational inference algorithms using optimal structured approximations for large sub-families of such models. The proposed sub-families will capture the properties of many important problems in the literature. Exploratory research in several specific applications further motivates the work and will be used to test the algorithms. The project will develop a new angle for theoretical analysis of Bayesian algorithms, deriving performance guarantees on their expected error. A core idea is to view variational inference algorithms through the so-called agnostic learning framework where guarantees sought are relative to the best that can be done within a specific limited class of approximations. This will provide a fresh outlook that informs the design of algorithms with desired performance guarantees. The expected scientific impact of the project is having better algorithms with well understood performance characteristics and applicable for a larger class of machine learning problems.

在过去的二十年里，贝叶斯模型已经成为机器学习的核心。贝叶斯模型通常假设潜在（未观察到的）变量对观察到的现象具有解释或预测能力。挑战在于从观察到的数据中推断出这些变量的状态或对该状态的信念。例如，可以尝试从关于用户自己的行为和其他用户的行为的观察来推断用户的偏好。该项目的目标是开发通用的近似推理算法，这些算法可以在大型贝叶斯模型家族中工作，以便解决方案可以广泛重用。算法工作将通过开发机器学习中的近似贝叶斯推理的学习理论来补充。该理论框架旨在证明贝叶斯预测算法的性能保证，并为具有理想特性的算法设计提供信息。该项目将有助于基础科研，推进机器学习的核心目标。该项目将支持博士生的培训和研究，因此将直接支持人类发展。通过课堂教学和外展，该项目将使更多的学生接触机器学习及其应用潜力。更具体地说，该项目将研究非共轭贝叶斯潜变量模型，即，它将避免经常使用但限制性的共轭性简化假设。在算法方面，该项目的目标是推广非共轭图形模型的变分消息传递的范例，并开发随机变分推理算法，使用最佳结构近似的大型子族的这种模型。所提出的子族将捕获文献中许多重要问题的属性。在几个特定应用程序的探索性研究进一步激发了工作，并将用于测试算法。该项目将为贝叶斯算法的理论分析开发一个新的角度，导出其预期误差的性能保证。一个核心思想是通过所谓的不可知学习框架来查看变分推理算法，其中所寻求的保证是相对于在特定的有限类近似中可以做到的最好的。这将提供一个新的前景，通知与期望的性能保证的算法的设计。该项目的预期科学影响是具有更好的算法，具有更好的性能特征，并适用于更大类别的机器学习问题。