权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Learning Structured Representations with Deep Probabilistic Programs

职业：通过深度概率程序学习结构化表示

基本信息

批准号：
2047253
负责人：
Jan-Willem van de Meent
金额：
$ 47.44万
依托单位：
Northeastern University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-10-01 至 2023-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047253&HistoricalAwards=false
关键词：
CAREER Learning Structured Representations Deep

项目摘要

Programming languages can play a decisive role in democratizing machine learning research. In deep learning, programming frameworks have made it possible – and even routine – to define neural networks in a modular manner. This has led to an explosion of research, with breakthroughs in computer vision, natural language processing, and reinforcement learning. The proposed work will develop deep probabilistic programming languages, which train neural networks to perform inference in simulation-based models. These languages will help the community address emerging challenges in artificial intelligence research by developing models that incorporate inductive biases to reason about uncertainty and improve generalization from limited data. In applications in the physical sciences, inductive biases can incorporate our physical knowledge of a problem domain. More generally, probabilistic programs help us represent model structure, for example to reason about how actions affect objects in a scene. The technical challenge that the proposed work addresses is scaling up methods for inference in probabilistic programs. To do so, the investigators will develop a language for inference programming, which will allow users to optimize the inference approach for a specific model. Inference methods reason about the posterior distribution over unknown variables in a program in light of observed data. Stochastic variational methods approximate the posterior by training a neural network that accepts data as input and returns a distribution over variables. This strategy works well in simple models in which unknown variables take the form of an unstructured vector. However, in models with more complex structure, efficient inference often requires reasoning about conditional independence. This is challenging for programmatically specified models, where reasoning about model structure requires program analysis. To address this challenge, the investigators will develop an inference language based on two constructs. The first are model combinators, which define a first-order language for composing black-box programs in a manner that allows us to reason about conditional independence. The second are inference combinators, which may be used to apply correct-by-construction importance sampling operations to specific components of the model. Together, model and inference combinators will allow users to develop correct and efficient stochastic variational methods for specific models. In addition to developing these fundamental abstractions and proving their correctness, the investigators will demonstrate the utility of these methods in applications to few-shot deep generative models, and structured energy-based models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

编程语言可以在机器学习研究的民主化中发挥决定性作用。在深度学习中，编程框架使以模块化方式定义神经网络成为可能--甚至是例行公事。这导致了研究的爆炸性增长，在计算机视觉、自然语言处理和强化学习方面取得了突破。这项拟议的工作将开发深度概率编程语言，这些语言训练神经网络在基于模拟的模型中执行推理。这些语言将帮助社区应对人工智能研究中新出现的挑战，方法是开发包含归纳偏见的模型，以推理不确定性，并从有限的数据中改进泛化。在物理科学的应用中，归纳偏差可以结合我们对问题领域的物理知识。更广泛地说，概率程序帮助我们表示模型结构，例如，推理动作如何影响场景中的对象。拟议的工作解决的技术挑战是扩大概率程序中的推理方法。为此，调查人员将开发一种推理编程语言，使用户能够针对特定模型优化推理方法。推理方法根据观测数据对程序中未知变量的后验分布进行推理。随机变分方法通过训练神经网络来逼近后验分布，神经网络接受数据作为输入，并返回变量的分布。这一策略在未知变量采用非结构化向量形式的简单模型中效果很好。然而，在结构更复杂的模型中，有效的推理通常需要关于条件独立性的推理。这对于以编程方式指定的模型是具有挑战性的，其中关于模型结构的推理需要程序分析。为了应对这一挑战，调查人员将开发一种基于两个结构的推理语言。第一种是模型组合器，它定义了一种用于编写黑盒程序的一阶语言，这种语言允许我们对条件独立性进行推理。第二种是推理合并器，它可用于对模型的特定组件应用逐个构造的重要性抽样操作。同时，模型和推理组合器将允许用户为特定模型开发正确和有效的随机变分方法。除了开发这些基本的抽象概念并证明它们的正确性外，研究人员还将展示这些方法在应用于少镜头深度生成模型和基于能量的结构化模型中的实用性。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。