Efficient Inference for Higher-Order Probabilistic Programs

研究计划背景：许多科学模型可以自然地表达为随机模拟器。概率编程允许用户利用这些模拟器的源代码信息进行贝叶斯推理。一般随机模拟器的全尺寸贝叶斯推理本质上为用户提供了一种基于观测数据反演模拟器的原则方法。例如，给定一个模拟疾病爆发的模拟器和一些观察到的数据，我们可以推断出最能描述给定疾病爆发的潜在参数。然而，贝叶斯统计中的大多数推理算法都是为具有固定维数的模型设计的。相反，高阶概率编程允许用户定义具有可变（甚至可能是无限）数量潜在变量的模型。高阶概率程序表示的模型的通用性要求设计新的推理算法，该算法必须具有足够的通用性，并能充分利用模拟器的程序结构。在这些模拟器中，高效和通用贝叶斯推理的潜在影响将是巨大的，因为它将允许建立精确模拟器的全新科学工作流程，这些模拟器可以根据观察到的数据进行反转和改进。目标和目标：基于源代码的静态分析，开发更适合特定概率程序的新型推理算法。将推理算法集成到流行的概率编程环境（如Pyro， Turing或PyProb）中，以便大量用户可以访问。研究方法的新新性：高阶概率程序的高效推理算法仍然是一个活跃的研究领域。目前的方法主要基于重要性抽样、顺序蒙特卡罗或变分推理。我们希望改进这些方法和/或潜在地解锁全新类型的推理算法。与EPSRC的战略和研究领域保持一致：人工智能技术编程语言和编译器统计和应用概率理论计算机科学

Context of the research proposal: Many scientific models can be naturally expressed as stochastic simulators. Probabilistic programming allows users to exploit the source code information of these simulators to conduct Bayesian inference. Full-scale Bayesian inference in general stochastic simulators essentially provides users with a principled way to invert simulators based on observed data. For example, given a simulator that models disease outbreaks and some observed data we can infer the underlying latent parameters which best describe the given disease outbreak. However, most inference algorithms in Bayesian statistics are designed for models which have a fixed dimensionality. In contrast, higher-order probabilistic programming allows the user to define models which have a variable (possibly even infinite) number of latent variables. The generality of models expressed in higher-order probabilistic programs requires the design of new inference algorithms which are sufficiently general and can exploit the program structure of the simulator. The potential impact of efficient and general Bayesian inference in these simulators would be enormous as it would allow for entirely new scientific workflows of building accurate simulators which can be inverted and improved based on observed data. Aims and objectives: Develop novel inference algorithms which are more tailored to a specific probabilistic program based on static analysis of source code Integrate inference algorithms within popular probabilistic programming environments such as Pyro, Turing or PyProb so that they are accessible to a large number of users Novelty of the research methodology: Efficient inference algorithms for higher-order probabilistic programs are still an active area of research. Current approaches are mostly based on Importance Sampling, Sequential Monte Carlo or Variational Inference. We hope to improve upon these approaches and/or potentially unlock entirely new types of inference algorithms. Alignment to EPSRC's strategies and research areas: Artificial Intelligence technologies Programming languages and compilers Statistic and applied probability Theoretical computer science