Efficient Inference for Higher-Order Probabilistic Programs

点击翻译按钮获取中文摘要

Context of the research proposal: Many scientific models can be naturally expressed as stochastic simulators. Probabilistic programming allows users to exploit the source code information of these simulators to conduct Bayesian inference. Full-scale Bayesian inference in general stochastic simulators essentially provides users with a principled way to invert simulators based on observed data. For example, given a simulator that models disease outbreaks and some observed data we can infer the underlying latent parameters which best describe the given disease outbreak. However, most inference algorithms in Bayesian statistics are designed for models which have a fixed dimensionality. In contrast, higher-order probabilistic programming allows the user to define models which have a variable (possibly even infinite) number of latent variables. The generality of models expressed in higher-order probabilistic programs requires the design of new inference algorithms which are sufficiently general and can exploit the program structure of the simulator. The potential impact of efficient and general Bayesian inference in these simulators would be enormous as it would allow for entirely new scientific workflows of building accurate simulators which can be inverted and improved based on observed data. Aims and objectives: Develop novel inference algorithms which are more tailored to a specific probabilistic program based on static analysis of source code Integrate inference algorithms within popular probabilistic programming environments such as Pyro, Turing or PyProb so that they are accessible to a large number of users Novelty of the research methodology: Efficient inference algorithms for higher-order probabilistic programs are still an active area of research. Current approaches are mostly based on Importance Sampling, Sequential Monte Carlo or Variational Inference. We hope to improve upon these approaches and/or potentially unlock entirely new types of inference algorithms. Alignment to EPSRC's strategies and research areas: Artificial Intelligence technologies Programming languages and compilers Statistic and applied probability Theoretical computer science