Big Hypotheses: A Fully Parallelised Bayesian Inference Solution

大假设：完全并行的贝叶斯推理解决方案

• 批准号: EP/R018537/1
• 负责人: Simon Maskell
• 金额: $ 325.9万
• 依托单位: University of Liverpool
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FR018537%2F1
• 关键词: Big; Hypotheses; Fully; Parallelised; Bayesian

Bayesian inference is a process which allows us to extract information from data. The process uses prior knowledge articulated as statistical models for the data. We are focused on developing a transformational solution to Data Science problems that can be posed as such Bayesian inference tasks. An existing family of algorithms, called Markov chain Monte Carlo (MCMC) algorithms, offer a family of solutions that offer impressive accuracy but demand significant computational load. For a significant subset of the users of Data Science that we interact with, while the accuracy offered by MCMC is recognised as potentially transformational, the computational load is just too great for MCMC to be a practical alternative to existing approaches. These users include academics working in science (e.g., Physics, Chemistry, Biology and the social sciences) as well as government and industry (e.g., in the pharmaceutical, defence and manufacturing sectors). The problem is then how to make the accuracy offered by MCMC accessible at a fraction of the computational cost.The solution we propose is based on replacing MCMC with a more recently developed family of algorithms, Sequential Monte Carlo (SMC) samplers. While MCMC, at its heart, manipulates a single sampling process, SMC samplers are an inherently population-based algorithm that manipulates a population of samples. This makes SMC samplers well suited to the task of being implemented in a way that exploits parallel computational resources. It is therefore possible to use emerging hardware (e.g., Graphics Processor Units (GPUs), Field Programmable Gate Arrays (FPGAs) and Intel's Xeon Phis as well as High Performance Computing (HPC) clusters) to make SMC samplers run faster. Indeed, our recent work (which has had to remove some algorithmic bottlenecks before making the progress we have achieved) has shown that SMC samplers can offer accuracy similar to MCMC but with implementations that are better suited to such emerging hardware.The benefits of using an SMC sampler in place of MCMC go beyond those made possible by simply posing a (tough) parallel computing challenge. The parameters of an MCMC algorithm necessarily differ from those related to a SMC sampler. These differences offer opportunities for SMC samplers to be developed in directions that are not possible with MCMC. For example, SMC samplers, in contrast to MCMC algorithms, can be configured to exploit a memory of their historic behaviour and can be designed to smoothly transition between problems. It seems likely that by exploiting such opportunities, we will generate SMC samplers that can outperform MCMC even more than is possible by using parallelised implementations alone.Our interactions with users, our experience of parallelising SMC samplers and the preliminary results we have obtained when comparing SMC samplers and MCMC make us excited about the potential that SMC samplers offer as a "New Approach for Data Science".Our current work has only begun to explore the potential offered by SMC samplers. We perceive significant benefit could result from a larger programme of work that helps us understand the extent to which users will benefit from replacing MCMC with SMC samplers. We propose a programme of work that combines a focus on users' problems with a systematic investigation into the opportunities offered by SMC samplers.Our strategy for achieving impact comprises multiple tactics. Specifically, we will: use identified users to act as "evangelists" in each of their domains; work with our hardware-oriented partners to produce high-performance reference implementations; engage with the developer team for Stan (the most widely-used generic MCMC implementation); work with the Industrial Mathematics Knowledge Transfer Network and the Alan Turing Institute to engage with both users and other algorithmic developers.

贝叶斯推理是一个允许我们从数据中提取信息的过程。该过程使用先验知识作为数据的统计模型。我们专注于开发数据科学问题的转换解决方案，这些问题可以作为贝叶斯推理任务提出。现有的一系列算法，称为马尔可夫链蒙特卡罗（MCMC）算法，提供了一系列解决方案，提供了令人印象深刻的准确性，但需要大量的计算负载。对于与我们互动的数据科学用户的一个重要子集，虽然MCMC提供的准确性被认为是潜在的转型，但MCMC的计算负荷太大，无法成为现有方法的实际替代方案。这些用户包括在科学领域（如物理、化学、生物和社会科学）以及政府和工业领域（如制药、国防和制造业）工作的学者。接下来的问题是如何在计算成本的一小部分上实现MCMC提供的精度。我们提出的解决方案是基于用最近开发的一系列算法取代MCMC，即顺序蒙特卡罗（SMC）采样器。虽然MCMC的核心是操纵单个采样过程，但SMC采样器本质上是一种基于种群的算法，可以操纵样本的总体。这使得SMC采样器非常适合以利用并行计算资源的方式实现的任务。因此，可以使用新兴硬件（例如，图形处理器单元（gpu），现场可编程门阵列（fpga）和英特尔的至强处理器以及高性能计算（HPC）集群）使SMC采样器运行得更快。事实上，我们最近的工作（在取得进展之前必须消除一些算法瓶颈）表明，SMC采样器可以提供与MCMC相似的精度，但其实现更适合此类新兴硬件。使用SMC采样器代替MCMC的好处远远超过了简单地提出（艰难的）并行计算挑战所带来的好处。MCMC算法的参数必然不同于与SMC采样器相关的参数。这些差异为SMC采样器在MCMC无法实现的方向上的发展提供了机会。例如，与MCMC算法相比，SMC采样器可以配置为利用其历史行为的记忆，并且可以设计为在问题之间顺利过渡。似乎有可能通过利用这些机会，我们将产生比单独使用并行实现更能超越MCMC的SMC采样器。我们与用户的互动，我们并行化SMC采样器的经验，以及我们在比较SMC采样器和MCMC时获得的初步结果，使我们对SMC采样器作为“数据科学的新方法”提供的潜力感到兴奋。我们目前的工作才刚刚开始探索SMC采样器提供的潜力。我们认为，一个更大的工作计划可能会带来显著的好处，这有助于我们了解用户将在多大程度上受益于用SMC采样器取代MCMC。我们提出了一项工作计划，将对用户问题的关注与对SMC采样器提供的机会的系统调查相结合。我们实现影响的战略包括多种策略。具体来说，我们将：使用已识别的用户在他们的每个领域充当“布道者”；与面向硬件的合作伙伴合作，生产高性能的参考实现；与Stan（最广泛使用的通用MCMC实现）的开发团队合作；与工业数学知识转移网络和艾伦图灵研究所合作，与用户和其他算法开发人员合作。

项目成果

Control Variates for Constrained Variables

约束变量的控制变量

• DOI: 10.1109/lsp.2022.3221347
• 发表时间: 2022
• 期刊: IEEE Signal Processing Letters
• 影响因子: 3.9
• 作者: Maskell S

Refining epidemiological forecasts with simple scoring rules.

使用简单的评分规则来完善流行病学预测。

• DOI: 10.1098/rsta.2021.0305
• 发表时间: 2022-10-03
• 期刊: PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
• 影响因子: 5
• 作者: Moore, Robert E.;Rosato, Conor;Maskell, Simon
• 通讯作者: Maskell, Simon

Extracting Self-Reported COVID-19 Symptom Tweets and Twitter Movement Mobility Origin/Destination Matrices to Inform Disease Models

• DOI: 10.3390/info14030170
• 发表时间: 2023-03-01
• 期刊: INFORMATION
• 影响因子: 3.1
• 作者: Rosato，Conor;Moore，Robert E. E.;Maskell，Simon
• 通讯作者: Maskell，Simon

Inference of Stochastic Disease Transmission Models Using Particle-MCMC and a Gradient Based Proposal

使用粒子 MCMC 和基于梯度的提案推断随机疾病传播模型

• DOI: 10.23919/fusion49751.2022.9841249
• 发表时间: 2022
• 作者: Rosato C

Refining Epidemiological Forecasts with Simple Scoring Rules

通过简单的评分规则完善流行病学预测

• DOI: 10.48550/arxiv.2111.04498
• 发表时间: 2021
• 作者: Moore R