Advanced Probabilistic Programming

概率编程语言（PPL）在表达和解决广泛的基于模型的统计推理问题方面正处于实用的前沿。我提出要验证的高级假设是，持续的PPL研究和开发将使人工智能（AI）社区有可能快速开发出用于感知、推理和行动选择的关键新概率模型，这些模型远远超出了当前深度学习系统的能力。******深度学习的基础由监督学习、大数据、快速计算机、神经网络架构和差异化自动化软件工具组成。我在这里提出的建议与开发用于自动微分的编程语言工具之间进行了类比。这些工具使学者和实践者更容易快速、轻松地实验新的神经网络架构，从而引发了深度学习革命。****** ppp的目标是在无监督学习和推理中发挥同样的作用。ppl包括贝叶斯网络、图形模型、因子图、贝叶斯非参数模型和贝叶斯深度学习模型中的外延和自动推理。不幸的是，直到最近，这还要求ppp使用能够“解决”“所有”推理问题的非常通用的推理算法。******该提案的主要部分是推进所谓的“推理编译”，这是一种最近引入的技术，它在概率编程和深度学习之间架起了桥梁，利用深度学习技术在结构丰富的PPL模型中显著加快平摊推理。总体目标是开发理论和软件，允许a)从PPL模型中推导出所谓的推理网络（IN）架构的结构（反之亦然），以及b)有效地训练这种INs，以便它们执行快速推理。******本提案的第二部分解决了即使编写可解释的，详细的和准确的生成模型本身也非常困难的问题，因此我还将在PPL/推理编译框架内研究生成模型学习。******在我看来，自动化半监督和无监督、生成模型学习和编译/平摊推理的语言是下一个高级人工智能工具链的关键组成部分。从理论到实现，这项拟议的研究和开发旨在实现这些语言的实际和可扩展的实现，并演示如何将它们用于下一代人工智能应用，如全自动驾驶、无限制问答系统等**

Probabilistic programming languages (PPL) are on the cusp of becoming practically useful for expressing and solving a wide-range of model-based statistical reasoning problems. The high-level hypothesis I propose to test is that continuing PPL research and development will make it possible for the artificial intelligence (AI) community to rapidly develop key new probabilistic models for perception, reasoning, and action selection that go far beyond what current deep learning systems are thought to be capable of now. ******The foundation of deep learning consists of supervised learning, big data, fast computers, neural net architectures, and differentiation automation software tools. I draw an analogy between what I propose here and the development of such programming language tools for automating differentiation. Such tools arguably led to the deep learning revolution by making it easier for academics and practitioners both to quickly and easily experiment with novel neural net architectures. ******PPLs aim to play the same role for unsupervised learning and inference. PPLs subsume the denotation of and automate inference in Bayes nets, graphical models, factor graphs, Bayesian nonparametric models, and Bayesian deep learning models. Until very recently this has unfortunately required PPLs to use very general purpose inference algorithms that are capable of “solving” “all” inference problems. ******The main part of this proposal is to advance so-called “inference compilation,” a very recently introduced technique that bridges between probabilistic programming and deep learning, leveraging deep learning techniques to dramatically speed amortized inference in richly structured PPL models. The general aim is to develop theory and software that allows a) derivation of the structure of so-called inference network (IN) architectures from PPL models (and vice versa) and b) efficient training of such INs so that they perform rapid inference. ******A second part of this proposal addresses the problem that even writing interpretable, detailed, and accurate generative models is itself very hard so I will also investigate generative model learning within the PPL/inference compilation framework. ******Languages that automate semi- and un-supervised, generative-model learning with compiled/amortized inference are, in my opinion, the key constituents of the next toolchain for advanced AI. From theory to implementation this proposed research and development aims for the realization of practical and scalable implementations of such languages and demonstrations of how they can be used for next-generation AI applications such as fully-autonomous driving, unrestricted question-answering systems, etc. **