Probabilistic programming for Bayesian nonparametrics

我的研究将集中在开发和应用统计机器学习技术。统计机器学习涉及建模和实现。从建模的角度来看，我将研究适合于异质数据的模型，比如认知科学中的数据。我还对如何对现实世界中出现的网络和图（社交网络、本体论图等）进行建模感兴趣。贝叶斯非参数模型是这类模型中的一大类，我很幸运能在贝叶斯非参数研究的前沿部门工作。训练贝叶斯非参数模型在开发时间和计算时间上都是复杂的。在我的项目的实现方面，我将研究概率规划对贝叶斯非参数模型的适用性。概率编程是一个快速发展的领域，涵盖了计算机科学、工程学、统计学和机器学习。虽然像Church这样的“第一代”语言主要用于研究目的，但更现代的概率语言，如STAN， Pyro和Edward，在一系列模型上提供高性能的概率推断。这在行业内得到了大量的吸收和发展（一个显著的例子是优步最近对概率编程的投资）。现有的概率语言已经完全集中在概率推理上。虽然我最初的工作（参见Bloem-Reddy、Mathieu、Foster、Rainforth、Teh、Ge、Lomeli和Ghahramani的《概率程序中离散随机概率度量的抽样和推理》）表明，贝叶斯非参数模型可以用现有的概率编程语言（在我们的例子中是图林）来表示，但要充分利用贝叶斯非参数的概率编程存在实质性障碍。我将研究的一个重要领域是模型中某些对称性的存在。考虑稀疏网络的贝叶斯非参数（参见Caron和Fox的“使用可交换随机度量的稀疏图”）。由于边缘的有限交换性，模型具有高度的对称性。然而，当天真地用概率语言表达这种模型时，对称性被打破，导致推理性能严重下降。与此密切相关的是在概率规划中处理维数随机变化的随机变量，或（等价地）存在性随机的变量。目前使用的大多数概率语言要么明确地排除这种设置，要么在这种设置中表现非常差。但这些正是概率规划对贝叶斯非参数模型有用所需要的设置。该项目的第二个目标是使概率编程超越推理。模型检查、模型选择、强化学习和控制的应用都是我打算解决的概率编程的令人兴奋的领域。就潜在影响而言，概率编程最紧迫的用例是在自动驾驶汽车领域。优步（Uber）和丰田（Toyota）最近在该领域的投资表明，工业界迫切需要在复杂模型上大规模实现概率编程。然而，概率编程是一个非常强大和通用的工具。来自网络（社交和其他）的数据现在遍布社会，对这些数据的有效分析对下一代政策制定者和数据分析师至关重要。概率编程将是他们武器库中的主要工具。从主题上讲，该项目属于以下EPSRC研究领域：人工智能技术，信息系统，统计和应用概率。

My research will be focused on developing and applying statistical machine learning techniques. Statistical machine learning is concerned with modelling and implementation. From a modelling perspective, I will work on models which are suitable for a heterogenous data such as data that arises in cognitive science. I am also interest in how we can model networks and graphs arising in real-world settings (social networks, ontological graphs, etc). Bayesian nonparametric models are a large class of such models, and I am fortunate to be working with in a department which is at the forefront of research in Bayesian nonparametrics. Training a Bayesian nonparametric model can be complex, both in development time and computation time. On the implementation side of my project, I will research the applicability of probabilistic programming to Bayesian nonparametric models. Probabilistic programming is a rapidly developing area that spans computer science, engineering, statistics and machine learning. Whilst 'generation 1' languages such as Church were primarily for research purposes only, more modern probabilistic languages like STAN, Pyro and Edward offer high- performance probabilistic inference on a range of models. There is substantial uptake and development within industry (a notable example being Uber's recent investment in probabilistic programming). Existing probabilistic languages have been squarely focused on probabilistic inference.Whilst my initial work (see 'Sampling and inference for discrete random probability measures in probabilistic programs' by Bloem-Reddy, Mathieu, Foster, Rainforth, Teh, Ge, Lomeli and Ghahramani) has shown that Bayesian nonparametric models can be expressed in an existing probabilistic programming language (in our case, Turing), there are substantial obstacles to a full use of probabilistic programming for Bayesian nonparametrics. One important area that I will examine is the existence of certain symmetries in the model. Consider Bayesian nonparametrics for sparse networks (see 'Sparse graphs using exchangeable random measures' by Caron and Fox). There is a high degree of symmetry in the model due to the finite edge exchangeability property. However, when naively expressing such a model in a probabilistic language, the symmetry is broken leading to severely diminished inference performance. Closely related to this is the treatment in probabilistic programming of random variable that change vary stochastically in dimension, or (equivalently) variables whose existence is stochastic. The majority of probabilistic languages in current use either explicitly exclude or perform very poorly in such settings. But these are precisely the settings needed for probabilistic programming to be useful for Bayesian nonparametric models. A second objective of the project is to take probabilistic programming beyond inference. Model checking, model selection, applications to reinforcement learning and control are all exciting areas for probabilistic programming that I intend to tackle.In terms of potential impact, the most pressing use case of probabilistic programming is in the field of autonomous vehicles.Recent investment by Uber and Toyota in the area indicates the burning need of industry to make probabilistic programming work at scale on sophisticated models. Probabilistic programming, though, is a very powerful and general tool. Data from networks (social and otherwise) now pervades society and efficient analysis of such data will be essential for the next generation of policy makers and data analysts.Probabilistic programming will be a major tool in their arsenal.Thematically, this project falls within the following EPSRC researchareas: Artificial intelligence technologies, Information systems, and Statistics and applied probability.