Advanced Probabilistic Programming

高级概率编程

• 批准号: RGPIN-2018-05022
• 负责人: Wood, Frank
• 金额: $ 4.01万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=713239
• 关键词: Advanced; Probabilistic; Programming

Probabilistic programming languages (PPL) are on the cusp of becoming practically useful for expressing and solving a wide-range of model-based statistical reasoning problems. The high-level hypothesis I propose to test is that continuing PPL research and development will make it possible for the artificial intelligence (AI) community to rapidly develop key new probabilistic models for perception, reasoning, and action selection that go far beyond what current deep learning systems are thought to be capable of now. The foundation of deep learning consists of supervised learning, big data, fast computers, neural net architectures, and differentiation automation software tools. I draw an analogy between what I propose here and the development of such programming language tools for automating differentiation. Such tools arguably led to the deep learning revolution by making it easier for academics and practitioners both to quickly and easily experiment with novel neural net architectures. PPLs aim to play the same role for unsupervised learning and inference. PPLs subsume the denotation of and automate inference in Bayes nets, graphical models, factor graphs, Bayesian nonparametric models, and Bayesian deep learning models. Until very recently this has unfortunately required PPLs to use very general purpose inference algorithms that are capable of “solving” “all” inference problems. The main part of this proposal is to advance so-called “inference compilation,” a very recently introduced technique that bridges between probabilistic programming and deep learning, leveraging deep learning techniques to dramatically speed amortized inference in richly structured PPL models. The general aim is to develop theory and software that allows a) derivation of the structure of so-called inference network (IN) architectures from PPL models (and vice versa) and b) efficient training of such INs so that they perform rapid inference. A second part of this proposal addresses the problem that even writing interpretable, detailed, and accurate generative models is itself very hard so I will also investigate generative model learning within the PPL/inference compilation framework. Languages that automate semi- and un-supervised, generative-model learning with compiled/amortized inference are, in my opinion, the key constituents of the next toolchain for advanced AI. From theory to implementation this proposed research and development aims for the realization of practical and scalable implementations of such languages and demonstrations of how they can be used for next-generation AI applications such as fully-autonomous driving, unrestricted question-answering systems, etc.

概率编程语言（PPL）在表达和解决各种基于模型的统计推理问题方面正变得越来越实用。我提出要测试的高级假设是，持续的PPL研究和开发将使人工智能（AI）社区有可能快速开发出用于感知、推理和动作选择的关键新概率模型，这些模型远远超出了目前深度学习系统的能力。 深度学习的基础包括监督学习、大数据、快速计算机、神经网络架构和微分自动化软件工具。我在这里提出的建议和开发这种自动微分的编程语言工具之间进行了类比。 这些工具可以说是导致深度学习革命，使学者和从业者更容易快速，轻松地实验新的神经网络架构。 PPL的目标是在无监督学习和推理中发挥同样的作用。PPL支持贝叶斯网络、图形模型、因子图、贝叶斯非参数模型和贝叶斯深度学习模型中的表示和自动推理。不幸的是，直到最近，这才要求PPL使用能够“解决”“所有”推理问题的非常通用的推理算法。 该提案的主要部分是推进所谓的“推理编译”，这是一种最近引入的技术，在概率编程和深度学习之间架起了桥梁，利用深度学习技术在结构丰富的PPL模型中显着加快摊销推理。总体目标是开发理论和软件，其允许a）从PPL模型导出所谓的推理网络（IN）架构的结构（反之亦然）和B）有效训练这样的IN，使得它们执行快速推理。 这个建议的第二部分解决了这样一个问题，即即使编写可解释的、详细的和准确的生成模型本身也是非常困难的，所以我还将在PPL/推理编译框架内研究生成模型学习。 在我看来，通过编译/摊销推理自动化半监督和无监督的生成模型学习的语言是高级AI下一个工具链的关键组成部分。 从理论到实现，这一拟议的研究和开发旨在实现这些语言的实用和可扩展的实现，并演示它们如何用于下一代人工智能应用，如全自动驾驶、无限制问答系统等。