Probabilistic programming for Bayesian nonparametrics

我的研究将集中在开发和应用统计机器学习技术。统计机器学习涉及建模和实施。从建模的角度来看，我将致力于适合于异质数据的模型，例如认知科学中产生的数据。我还对如何对真实世界环境(社交网络、本体图等)中出现的网络和图进行建模感兴趣。贝叶斯非参数模型是这类模型中的一大类，我很幸运地在一个处于贝叶斯非参数学研究前沿的部门工作。训练贝叶斯非参数模型可能是复杂的，无论是开发时间还是计算时间。在我的项目实施方面，我将研究概率规划对贝叶斯非参数模型的适用性。概率规划是一个迅速发展的领域，它横跨计算机科学、工程学、统计学和机器学习。虽然丘奇等第一代语言主要用于研究目的，但更现代的概率语言，如斯坦、皮罗和爱德华，则在一系列模型上提供高性能的概率推理。行业内部有很大的吸收和发展(一个明显的例子是优步最近在概率编程方面的投资)。虽然我最初的工作(见Bloem-Reddy，Mathieu，Foster，Rainforth，Teh，GE，Lomeli和Ghahramani的《概率程序中离散随机概率度量的抽样和推断》)已经表明贝叶斯非参数模型可以用现有的概率编程语言(在我们的例子中是图灵)来表示，但是充分使用贝叶斯非参数的概率编程是有实质性的障碍的。我将研究的一个重要领域是模型中某些对称性的存在。考虑稀疏网络的贝叶斯非参数(参见Caron和Fox的《使用可交换随机措施的稀疏图》)。由于有限边的可交换性，模型具有高度的对称性。然而，当用概率语言幼稚地表达这样的模型时，对称性被打破，导致推理性能严重降低。与此密切相关的是在概率规划中对随机变量的处理，这些随机变量的变化在维度上随机变化，或者(等价地)存在随机变量。当前使用的大多数概率语言要么明确排除，要么在这样的环境中表现得非常差。但这些正是概率编程对贝叶斯非参数模型有用所需的设置。该项目的第二个目标是使概率编程超越推理。模型检查、模型选择、强化学习和控制的应用都是我打算解决的概率编程令人兴奋的领域。就潜在影响而言，概率编程最紧迫的用例是在自动驾驶汽车领域。优步和丰田最近在该领域的投资表明，行业迫切需要在复杂模型上进行大规模的概率编程工作。然而，概率编程是一个非常强大和通用的工具。来自网络(社会和其他)的数据现在弥漫在社会中，对这些数据的有效分析将对下一代政策制定者和数据分析员至关重要。概率编程将是他们的主要工具。从理论上讲，这个项目属于EPSRC的以下研究领域：人工智能技术、信息系统、统计和应用概率。

My research will be focused on developing and applying statistical machine learning techniques. Statistical machine learning is concerned with modelling and implementation. From a modelling perspective, I will work on models which are suitable for a heterogenous data such as data that arises in cognitive science. I am also interest in how we can model networks and graphs arising in real-world settings (social networks, ontological graphs, etc). Bayesian nonparametric models are a large class of such models, and I am fortunate to be working with in a department which is at the forefront of research in Bayesian nonparametrics. Training a Bayesian nonparametric model can be complex, both in development time and computation time. On the implementation side of my project, I will research the applicability of probabilistic programming to Bayesian nonparametric models. Probabilistic programming is a rapidly developing area that spans computer science, engineering, statistics and machine learning. Whilst 'generation 1' languages such as Church were primarily for research purposes only, more modern probabilistic languages like STAN, Pyro and Edward offer high- performance probabilistic inference on a range of models. There is substantial uptake and development within industry (a notable example being Uber's recent investment in probabilistic programming). Existing probabilistic languages have been squarely focused on probabilistic inference.Whilst my initial work (see 'Sampling and inference for discrete random probability measures in probabilistic programs' by Bloem-Reddy, Mathieu, Foster, Rainforth, Teh, Ge, Lomeli and Ghahramani) has shown that Bayesian nonparametric models can be expressed in an existing probabilistic programming language (in our case, Turing), there are substantial obstacles to a full use of probabilistic programming for Bayesian nonparametrics. One important area that I will examine is the existence of certain symmetries in the model. Consider Bayesian nonparametrics for sparse networks (see 'Sparse graphs using exchangeable random measures' by Caron and Fox). There is a high degree of symmetry in the model due to the finite edge exchangeability property. However, when naively expressing such a model in a probabilistic language, the symmetry is broken leading to severely diminished inference performance. Closely related to this is the treatment in probabilistic programming of random variable that change vary stochastically in dimension, or (equivalently) variables whose existence is stochastic. The majority of probabilistic languages in current use either explicitly exclude or perform very poorly in such settings. But these are precisely the settings needed for probabilistic programming to be useful for Bayesian nonparametric models. A second objective of the project is to take probabilistic programming beyond inference. Model checking, model selection, applications to reinforcement learning and control are all exciting areas for probabilistic programming that I intend to tackle.In terms of potential impact, the most pressing use case of probabilistic programming is in the field of autonomous vehicles.Recent investment by Uber and Toyota in the area indicates the burning need of industry to make probabilistic programming work at scale on sophisticated models. Probabilistic programming, though, is a very powerful and general tool. Data from networks (social and otherwise) now pervades society and efficient analysis of such data will be essential for the next generation of policy makers and data analysts.Probabilistic programming will be a major tool in their arsenal.Thematically, this project falls within the following EPSRC researchareas: Artificial intelligence technologies, Information systems, and Statistics and applied probability.