Big Hypotheses: A Fully Parallelised Bayesian Inference Solution

贝叶斯推理是一个允许我们从数据中提取信息的过程。这一过程使用作为数据统计模型表述的先验知识。我们专注于为数据科学问题开发一个变革性的解决方案，可以被认为是这样的贝叶斯推理任务。现有的一系列算法，称为马尔可夫链蒙特卡罗(MCMC)算法，提供了一系列解决方案，提供了令人印象深刻的精度，但需要大量的计算负载。对于我们与之交互的数据科学用户中的一个重要子集来说，尽管MCMC提供的准确性被认为具有潜在的变革性，但对于MCMC来说，计算负荷太大了，无法成为现有方法的实际替代方案。这些用户包括从事科学(例如，物理、化学、生物学和社会科学)工作的学者以及政府和工业(例如，在制药、国防和制造部门)。我们提出的解决方案是用最近发展起来的一系列算法--序贯蒙特卡罗(SMC)采样器来代替MCMC。虽然MCMC的核心是操纵单个采样过程，但SMC采样器本质上是一种基于总体的算法，可以操纵总体样本。这使得SMC采样器非常适合以利用并行计算资源的方式实现的任务。因此，可以使用新兴硬件(例如图形处理器单元(GPU)、现场可编程门阵列(FPGA)和英特尔的至强PHI以及高性能计算(HPC)集群)来使SMC采样器运行得更快。事实上，我们最近的工作(在取得进展之前必须消除一些算法瓶颈)表明，SMC采样器可以提供类似于MCMC的精度，但其实现更适合于这种新兴的硬件。使用SMC采样器取代MCMC的好处超过了通过简单地提出(困难的)并行计算挑战而实现的那些。MCMC算法的参数必然不同于与SMC采样器相关的参数。这些不同之处为SMC采样器提供了机会，使其朝着MCMC无法实现的方向发展。例如，与MCMC算法不同，SMC采样器可以配置为利用其历史行为的记忆，并可以设计为在问题之间平稳过渡。通过利用这样的机会，我们很可能会产生比单独使用并行实现更好的SMC采样器。我们与用户的交互、我们并行SMC采样器的经验以及我们在比较SMC采样器和MCMC时获得的初步结果，使我们对SMC采样器提供的“数据科学新方法”的潜力感到兴奋。我们目前的工作才刚刚开始探索SMC采样器提供的潜力。我们认为，一个更大的工作计划可能会带来显著的好处，帮助我们了解用户将在多大程度上受益于用SMC取样器取代MCMC。我们提出了一项工作计划，将对用户问题的关注与对SMC采样器提供的机会的系统调查相结合。我们实现影响的战略包括多种策略。具体地说，我们将：使用确定的用户在他们的每个领域中充当“布道者”；与我们面向硬件的合作伙伴合作，以产生高性能的参考实现；与STAN(最广泛使用的通用MCMC实现)的开发团队合作；与工业数学知识转移网络和艾伦·图灵研究所合作，与用户和其他算法开发人员合作。

Bayesian inference is a process which allows us to extract information from data. The process uses prior knowledge articulated as statistical models for the data. We are focused on developing a transformational solution to Data Science problems that can be posed as such Bayesian inference tasks. An existing family of algorithms, called Markov chain Monte Carlo (MCMC) algorithms, offer a family of solutions that offer impressive accuracy but demand significant computational load. For a significant subset of the users of Data Science that we interact with, while the accuracy offered by MCMC is recognised as potentially transformational, the computational load is just too great for MCMC to be a practical alternative to existing approaches. These users include academics working in science (e.g., Physics, Chemistry, Biology and the social sciences) as well as government and industry (e.g., in the pharmaceutical, defence and manufacturing sectors). The problem is then how to make the accuracy offered by MCMC accessible at a fraction of the computational cost.The solution we propose is based on replacing MCMC with a more recently developed family of algorithms, Sequential Monte Carlo (SMC) samplers. While MCMC, at its heart, manipulates a single sampling process, SMC samplers are an inherently population-based algorithm that manipulates a population of samples. This makes SMC samplers well suited to the task of being implemented in a way that exploits parallel computational resources. It is therefore possible to use emerging hardware (e.g., Graphics Processor Units (GPUs), Field Programmable Gate Arrays (FPGAs) and Intel's Xeon Phis as well as High Performance Computing (HPC) clusters) to make SMC samplers run faster. Indeed, our recent work (which has had to remove some algorithmic bottlenecks before making the progress we have achieved) has shown that SMC samplers can offer accuracy similar to MCMC but with implementations that are better suited to such emerging hardware.The benefits of using an SMC sampler in place of MCMC go beyond those made possible by simply posing a (tough) parallel computing challenge. The parameters of an MCMC algorithm necessarily differ from those related to a SMC sampler. These differences offer opportunities for SMC samplers to be developed in directions that are not possible with MCMC. For example, SMC samplers, in contrast to MCMC algorithms, can be configured to exploit a memory of their historic behaviour and can be designed to smoothly transition between problems. It seems likely that by exploiting such opportunities, we will generate SMC samplers that can outperform MCMC even more than is possible by using parallelised implementations alone.Our interactions with users, our experience of parallelising SMC samplers and the preliminary results we have obtained when comparing SMC samplers and MCMC make us excited about the potential that SMC samplers offer as a "New Approach for Data Science".Our current work has only begun to explore the potential offered by SMC samplers. We perceive significant benefit could result from a larger programme of work that helps us understand the extent to which users will benefit from replacing MCMC with SMC samplers. We propose a programme of work that combines a focus on users' problems with a systematic investigation into the opportunities offered by SMC samplers.Our strategy for achieving impact comprises multiple tactics. Specifically, we will: use identified users to act as "evangelists" in each of their domains; work with our hardware-oriented partners to produce high-performance reference implementations; engage with the developer team for Stan (the most widely-used generic MCMC implementation); work with the Industrial Mathematics Knowledge Transfer Network and the Alan Turing Institute to engage with both users and other algorithmic developers.