Big Hypotheses: A Fully Parallelised Bayesian Inference Solution

贝叶斯推理是一个允许我们从数据中提取信息的过程。该过程使用作为数据的统计模型表达的先验知识。我们专注于为数据科学问题开发一个转型解决方案，这些问题可以被视为贝叶斯推理任务。一个现有的算法家族，称为马尔可夫链蒙特卡罗（MCMC）算法，提供了一个家庭的解决方案，提供令人印象深刻的准确性，但需要显着的计算负载。对于我们与之互动的数据科学用户的重要子集来说，虽然MCMC提供的准确性被认为是潜在的变革，但MCMC的计算负荷太大，无法成为现有方法的实际替代方案。这些用户包括从事科学工作的学者（例如，物理学、化学、生物学和社会科学）以及政府和工业（例如，在制药、国防和制造业部门）。然后，问题是如何使MCMC提供的准确性，在一小部分的计算cost.The解决方案，我们提出的是基于替代MCMC与最近开发的算法，顺序蒙特卡罗（SMC）采样家庭。MCMC的核心是操纵单个采样过程，而SMC采样器是一种固有的基于种群的算法，可以操纵样本种群。这使得SMC采样器非常适合以利用并行计算资源的方式实现的任务。因此可以使用新兴的硬件（例如，图形处理器单元（GPU）、现场可编程门阵列（FPGA）和英特尔的至强融核以及高性能计算（HPC）集群），使SMC采样器运行速度更快。事实上，我们最近的工作（在取得进展之前必须消除一些算法瓶颈）表明，SMC采样器可以提供类似于MCMC的精度，但其实现更适合这种新兴的硬件。使用SMC采样器代替MCMC的好处超出了那些可能通过简单地提出（坚韧的）并行计算挑战。MCMC算法的参数必然不同于SMC采样器的参数。这些差异为SMC采样器提供了在MCMC不可能的方向上开发的机会。例如，与MCMC算法相比，SMC采样器可以被配置为利用其历史行为的记忆，并且可以被设计为在问题之间平滑过渡。通过利用这些机会，我们很可能会生成比单独使用并行实现更好的SMC采样器。我们与用户的交互，我们在并行化SMC采样器方面的经验，以及我们在比较SMC采样器和MCMC时获得的初步结果，使我们对SMC采样器作为“数据科学新方法”的潜力感到兴奋。我们目前的工作只是开始探索SMC采样器提供的潜力。我们认为，一个更大的工作方案可能会带来重大利益，这有助于我们了解用户将在多大程度上受益于用SMC采样器取代MCMC。我们提出了一个工作方案，将对用户问题的关注与对SMC采样器提供的机会的系统调查相结合。具体而言，我们将：我们使用已识别的用户在其每个领域充当“传道者”;与我们面向硬件的合作伙伴合作，以产生高性能的参考实现;与Stan（最广泛使用的通用MCMC实现）的开发团队合作;与工业数学知识转移网络和艾伦图灵研究所合作，与用户和其他算法开发人员合作。

Bayesian inference is a process which allows us to extract information from data. The process uses prior knowledge articulated as statistical models for the data. We are focused on developing a transformational solution to Data Science problems that can be posed as such Bayesian inference tasks. An existing family of algorithms, called Markov chain Monte Carlo (MCMC) algorithms, offer a family of solutions that offer impressive accuracy but demand significant computational load. For a significant subset of the users of Data Science that we interact with, while the accuracy offered by MCMC is recognised as potentially transformational, the computational load is just too great for MCMC to be a practical alternative to existing approaches. These users include academics working in science (e.g., Physics, Chemistry, Biology and the social sciences) as well as government and industry (e.g., in the pharmaceutical, defence and manufacturing sectors). The problem is then how to make the accuracy offered by MCMC accessible at a fraction of the computational cost.The solution we propose is based on replacing MCMC with a more recently developed family of algorithms, Sequential Monte Carlo (SMC) samplers. While MCMC, at its heart, manipulates a single sampling process, SMC samplers are an inherently population-based algorithm that manipulates a population of samples. This makes SMC samplers well suited to the task of being implemented in a way that exploits parallel computational resources. It is therefore possible to use emerging hardware (e.g., Graphics Processor Units (GPUs), Field Programmable Gate Arrays (FPGAs) and Intel's Xeon Phis as well as High Performance Computing (HPC) clusters) to make SMC samplers run faster. Indeed, our recent work (which has had to remove some algorithmic bottlenecks before making the progress we have achieved) has shown that SMC samplers can offer accuracy similar to MCMC but with implementations that are better suited to such emerging hardware.The benefits of using an SMC sampler in place of MCMC go beyond those made possible by simply posing a (tough) parallel computing challenge. The parameters of an MCMC algorithm necessarily differ from those related to a SMC sampler. These differences offer opportunities for SMC samplers to be developed in directions that are not possible with MCMC. For example, SMC samplers, in contrast to MCMC algorithms, can be configured to exploit a memory of their historic behaviour and can be designed to smoothly transition between problems. It seems likely that by exploiting such opportunities, we will generate SMC samplers that can outperform MCMC even more than is possible by using parallelised implementations alone.Our interactions with users, our experience of parallelising SMC samplers and the preliminary results we have obtained when comparing SMC samplers and MCMC make us excited about the potential that SMC samplers offer as a "New Approach for Data Science".Our current work has only begun to explore the potential offered by SMC samplers. We perceive significant benefit could result from a larger programme of work that helps us understand the extent to which users will benefit from replacing MCMC with SMC samplers. We propose a programme of work that combines a focus on users' problems with a systematic investigation into the opportunities offered by SMC samplers.Our strategy for achieving impact comprises multiple tactics. Specifically, we will: use identified users to act as "evangelists" in each of their domains; work with our hardware-oriented partners to produce high-performance reference implementations; engage with the developer team for Stan (the most widely-used generic MCMC implementation); work with the Industrial Mathematics Knowledge Transfer Network and the Alan Turing Institute to engage with both users and other algorithmic developers.