Advanced Probabilistic Programming

概率编程语言(PPL)即将成为表达和解决广泛的基于模型的统计推理问题的实用工具。我提议测试的高级假设是，持续的PPL研究和开发将使人工智能(AI)社区有可能迅速开发用于感知、推理和行动选择的关键新概率模型，这些模型远远超出当前深度学习系统目前被认为能够实现的能力。*深度学习的基础包括监督学习、大数据、快速计算机、神经网络结构和差异化自动化软件工具。我将我在这里提出的建议与这种用于自动区分的编程语言工具的开发进行类比。这些工具可以说是导致了深度学习革命，使学者和实践者更容易快速和轻松地试验新的神经网络结构。*ppls的目标是在无监督学习和推理方面发挥同样的作用。PLS包括贝叶斯网、图形模型、因子图、贝叶斯非参数模型和贝叶斯深度学习模型的表示和自动推理。不幸的是，直到最近，这还需要PPL使用能够“解决”“所有”推理问题的非常通用的推理算法。*这项建议的主要部分是推进所谓的“推理编译”，这是一种最近引入的技术，它在概率编程和深度学习之间架起了桥梁，利用深度学习技术在结构丰富的PPL模型中显著加快了分期推理的速度。总体目标是开发理论和软件，以允许a)从PPL模型推导出所谓的推理网络(IN)结构的结构(反之亦然)，以及b)有效地训练这种推理网络，以便它们执行快速推理。*本提案的第二部分解决了即使编写可解释的、详细的和准确的生成模型本身也非常困难的问题，因此我还将在PPL/推理编译框架内研究生成模型学习。*通过编译/摊销推理自动执行半监督和无监督的生成性模型学习的语言，在我看来，是高级人工智能下一代工具链的关键组成部分。从理论到实现，这项拟议的研究和开发旨在实现此类语言的实用和可扩展实现，并演示如何将它们用于下一代人工智能应用，如全自动驾驶、无限制问答系统等。**

Probabilistic programming languages (PPL) are on the cusp of becoming practically useful for expressing and solving a wide-range of model-based statistical reasoning problems. The high-level hypothesis I propose to test is that continuing PPL research and development will make it possible for the artificial intelligence (AI) community to rapidly develop key new probabilistic models for perception, reasoning, and action selection that go far beyond what current deep learning systems are thought to be capable of now. ******The foundation of deep learning consists of supervised learning, big data, fast computers, neural net architectures, and differentiation automation software tools. I draw an analogy between what I propose here and the development of such programming language tools for automating differentiation. Such tools arguably led to the deep learning revolution by making it easier for academics and practitioners both to quickly and easily experiment with novel neural net architectures. ******PPLs aim to play the same role for unsupervised learning and inference. PPLs subsume the denotation of and automate inference in Bayes nets, graphical models, factor graphs, Bayesian nonparametric models, and Bayesian deep learning models. Until very recently this has unfortunately required PPLs to use very general purpose inference algorithms that are capable of “solving” “all” inference problems. ******The main part of this proposal is to advance so-called “inference compilation,” a very recently introduced technique that bridges between probabilistic programming and deep learning, leveraging deep learning techniques to dramatically speed amortized inference in richly structured PPL models. The general aim is to develop theory and software that allows a) derivation of the structure of so-called inference network (IN) architectures from PPL models (and vice versa) and b) efficient training of such INs so that they perform rapid inference. ******A second part of this proposal addresses the problem that even writing interpretable, detailed, and accurate generative models is itself very hard so I will also investigate generative model learning within the PPL/inference compilation framework. ******Languages that automate semi- and un-supervised, generative-model learning with compiled/amortized inference are, in my opinion, the key constituents of the next toolchain for advanced AI. From theory to implementation this proposed research and development aims for the realization of practical and scalable implementations of such languages and demonstrations of how they can be used for next-generation AI applications such as fully-autonomous driving, unrestricted question-answering systems, etc. **