Optimisation-centric Generalisations of Bayesian Inference

以优化为中心的贝叶斯推理推广

• 批准号: EP/W005859/1
• 负责人: Jeremias Knoblauch
• 金额: $ 41.5万
• 依托单位: University College London
• 依托单位国家: 英国
• 项目类别: Fellowship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FW005859%2F1
• 关键词: Optimisation; centric; Generalisations; Bayesian; Inference

Large scale black box statistical models are ubiquitous in modern society; and aimed at providing a way to examine the behaviour of complex systems. For example, Improbable has helped design such models as part of the RAMP initiative to help the UK government predict the spread of the COVID-19 virus. In engineering, so-called 'digital twins' of real-world physical phenomena or assets are commonly used to conduct virtual stress tests and predict the behaviour of critical systems in the presence of exogenous shocks.An important concern for these models is the nature of our uncertainty about their predictions and recommendations. Unlike for more traditional statistical analysis, the underlying models are often highly complex, not immediately interpretable, and often misspecified. As a consequence, standard Bayesian methods of uncertainty quantification derived under the assumptions of the traditional paradigm for statistical analysis are often inappropriate. More specifically, they often result in over-confidence and a lack of robustness. To tackle this issue, generalised forms of Bayesian uncertainty quantification have recently been developed. Such methods can ensure robustness and reduce the computational burden relative to standard Bayesian methods. This makes them ideal for applications in simulation-based modelling scenarios---such as COVID-19 modelling or digital twins. Yet, to date they have not been used in this context and primarily enjoyed success in time-ordered problems (such as on-line learning, changepoint detection, or filtering and smoothing) as well as in Bayesian Deep Learning applications (such as Bayesian neural networks or deep Gaussian Processes). In spite of their promise however, both their foundational theoretical properties as well as their computation are under-explored topics of research. In this fellowship, I will advance the theory, methodology, and application of generalised Bayesian posteriors that are defined implicitly through an optimisation problem. While such generalised Bayesian methods have shown great promise, a thorough investigation of this kind will be required if they are to be adopted more widely. As part of this, I will investigate the fundamental question of how one should choose between different generalised posteriors. Complementing this, I will devise methodology for Bayesian computation geared towards the special properties of these posteriors. I will then leverage the advances made as part of this research to apply them on two classes of high-impact problems that traditional Bayesian methods struggle with: models revolving around intractable likelihoods, and simulator-based inference. For the applied component of this research programme, I will draw on the expertise of my project partners and use generalised posteriors for better uncertainty quantification in 'digital twins', as well as applications of importance for national security---such as modelling the COVID-19 pandemic.

大规模黑箱统计模型在现代社会中无处不在，旨在提供一种检查复杂系统行为的方法。例如，Improbable帮助设计了此类模型，作为RAMP计划的一部分，以帮助英国政府预测COVID-19病毒的传播。在工程领域，现实世界物理现象或资产的所谓“数字孪生”通常用于进行虚拟压力测试，并预测存在外部冲击时关键系统的行为。这些模型的一个重要问题是我们对其预测和建议的不确定性。与更传统的统计分析不同，底层模型通常非常复杂，无法立即解释，并且经常被错误指定。因此，在传统统计分析范式的假设下得出的不确定性量化的标准贝叶斯方法往往是不合适的。更具体地说，它们往往导致过度自信和缺乏鲁棒性。 为了解决这个问题，最近开发了贝叶斯不确定性量化的一般形式。相对于标准贝叶斯方法，这样的方法可以确保鲁棒性并减少计算负担。这使得它们非常适合基于模拟的建模场景中的应用-例如COVID-19建模或数字孪生模型。然而，到目前为止，它们还没有在这种情况下使用，主要在时间排序问题（如在线学习，变点检测或滤波和平滑）以及贝叶斯深度学习应用（如贝叶斯神经网络或深度高斯过程）中取得成功。尽管他们的承诺，但是，他们的基本理论性质以及他们的计算是未开发的研究课题。在这个奖学金，我将推进的理论，方法和广义贝叶斯后验的应用，是通过一个优化问题隐含定义。虽然这种广义贝叶斯方法已经显示出很大的希望，这种彻底的调查将需要，如果他们被更广泛地采用。作为其中的一部分，我将研究如何在不同的广义后验之间进行选择的基本问题。补充这一点，我将设计面向这些后验的特殊属性的贝叶斯计算方法。然后，我将利用作为本研究的一部分所取得的进展，将其应用于传统贝叶斯方法难以解决的两类高影响力问题：围绕棘手的可能性的模型和基于模拟器的推理。对于这个研究项目的应用部分，我将利用我的项目合作伙伴的专业知识，并使用广义后验方法来更好地量化“数字孪生”中的不确定性，以及对国家安全具有重要意义的应用-例如COVID-19大流行的建模。

项目成果

A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods

• DOI: 10.48550/arxiv.2305.15027
• 发表时间: 2023-05
• 期刊: ArXiv
• 作者: Veit Wild;Sahra Ghalebikesabi;D. Sejdinovic;Jeremias Knoblauch

Adversarial Interpretation of Bayesian Inference

• 发表时间: 2022
• 作者: Hisham Husain;Jeremias Knoblauch

Robust generalised Bayesian inference for intractable likelihoods

• DOI: 10.1111/rssb.12500
• 发表时间: 2021-04
• 期刊: Journal of the Royal Statistical Society: Series B (Statistical Methodology)
• 作者: Takuo Matsubara;Jeremias Knoblauch;François‐Xavier Briol;C. Oates

Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap

• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Charita Dellaporta;Jeremias Knoblauch;T. Damoulas;F. Briol

An Optimization-centric View on Bayes' Rule: Reviewing and Generalizing Variational Inference

• 发表时间: 2022
• 期刊: J. Mach. Learn. Res.
• 作者: Jeremias Knoblauch;Jack Jewson;T. Damoulas