Probabilistic programming for Bayesian nonparametrics

点击翻译按钮获取中文摘要

My research will be focused on developing and applying statistical machine learning techniques. Statistical machine learning is concerned with modelling and implementation. From a modelling perspective, I will work on models which are suitable for a heterogenous data such as data that arises in cognitive science. I am also interest in how we can model networks and graphs arising in real-world settings (social networks, ontological graphs, etc). Bayesian nonparametric models are a large class of such models, and I am fortunate to be working with in a department which is at the forefront of research in Bayesian nonparametrics. Training a Bayesian nonparametric model can be complex, both in development time and computation time. On the implementation side of my project, I will research the applicability of probabilistic programming to Bayesian nonparametric models. Probabilistic programming is a rapidly developing area that spans computer science, engineering, statistics and machine learning. Whilst 'generation 1' languages such as Church were primarily for research purposes only, more modern probabilistic languages like STAN, Pyro and Edward offer high- performance probabilistic inference on a range of models. There is substantial uptake and development within industry (a notable example being Uber's recent investment in probabilistic programming). Existing probabilistic languages have been squarely focused on probabilistic inference.Whilst my initial work (see 'Sampling and inference for discrete random probability measures in probabilistic programs' by Bloem-Reddy, Mathieu, Foster, Rainforth, Teh, Ge, Lomeli and Ghahramani) has shown that Bayesian nonparametric models can be expressed in an existing probabilistic programming language (in our case, Turing), there are substantial obstacles to a full use of probabilistic programming for Bayesian nonparametrics. One important area that I will examine is the existence of certain symmetries in the model. Consider Bayesian nonparametrics for sparse networks (see 'Sparse graphs using exchangeable random measures' by Caron and Fox). There is a high degree of symmetry in the model due to the finite edge exchangeability property. However, when naively expressing such a model in a probabilistic language, the symmetry is broken leading to severely diminished inference performance. Closely related to this is the treatment in probabilistic programming of random variable that change vary stochastically in dimension, or (equivalently) variables whose existence is stochastic. The majority of probabilistic languages in current use either explicitly exclude or perform very poorly in such settings. But these are precisely the settings needed for probabilistic programming to be useful for Bayesian nonparametric models. A second objective of the project is to take probabilistic programming beyond inference. Model checking, model selection, applications to reinforcement learning and control are all exciting areas for probabilistic programming that I intend to tackle.In terms of potential impact, the most pressing use case of probabilistic programming is in the field of autonomous vehicles.Recent investment by Uber and Toyota in the area indicates the burning need of industry to make probabilistic programming work at scale on sophisticated models. Probabilistic programming, though, is a very powerful and general tool. Data from networks (social and otherwise) now pervades society and efficient analysis of such data will be essential for the next generation of policy makers and data analysts.Probabilistic programming will be a major tool in their arsenal.Thematically, this project falls within the following EPSRC researchareas: Artificial intelligence technologies, Information systems, and Statistics and applied probability.