Fast Bayesian Deep Learning

近年来，深度学习已经被证明在一系列应用领域提供了令人印象深刻的性能。在某种程度上，这是因为它能够利用现代多核处理器（例如gpu）。深度学习的核心是使用随机梯度上升，这种方法在局部最大值和（更多的）鞍点的情况下都很困难，使用固定数据集可能会降低性能。以马尔可夫链蒙特卡罗（MCMC）为代表的数值贝叶斯方法取代随机梯度上升的技术已经发展起来。不幸的是，虽然MCMC的这种使用使得理论上可以总是找到全局最大值，但MCMC速度很慢，因此在实践中很难实现。这限制了MCMC在深度学习环境中使用的程度。利物浦大学最近的工作开发了顺序蒙特卡罗（SMC）采样器的一种变体，作为MCMC的通用替代品。SMC采样器似乎提供了与MCMC相同的全局优化能力，但大大降低了计算成本。这在一定程度上是因为SMC采样器本质上是并行的，这使得利用gpu等设备成为可能。深度学习通过在数据中分布处理来实现并行性。SMC采样器通过分布考虑数据可能暗示为真的许多假设来实现并行性。该学生将首先研究深度学习和SMC采样器的组合可以通过同时考虑数据和假设的并行性来并行化的程度。然后，学生将继续进行只有在快速贝叶斯深度学习（例如，与鲁棒异常值检测相关）存在的情况下才有可能进行的研究。

Over recent years, Deep Learning has been demonstrated to offer impressive performance across a range of application domains. In part, this is because of its ability to exploit modern many-cored processors (e.g., GPUs). At its heart, Deep Learning makes use of stochastic gradient ascent, an approach that is known to struggle in the context of both local maxima and (the more numerous) saddle-points, degrading performance possible using a fixed dataset. Techniques have been developed that replace the stochastic gradient ascent with numerical Bayesian methods typified by Markov chain Monte Carlo (MCMC). Unfortunately, while such use of MCMC makes it theoretically possible to always find the global maximum, MCMC is so slow that this is hard to achieve in practice. This has restricted the extent to which MCMC has been used in the context of Deep Learning. Recent work by the University of Liverpool has developed a variant of the Sequential Monte Carlo (SMC) sampler, as a generic alternative to MCMC. SMC samplers appear to offer the same global optimisation capability as MCMC but with substantially reduced computational cost. This comes about, in part, because SMC samplers are inherently parallel, making it possible to exploit, for example, GPUs. Deep Learning achieves parallelism by distributing the processing across the data. SMC samplers achieve parallelism by distributing the consideration of the many hypotheses that the data could imply to be true. This studentship will begin by investigating the extent to which a combination of Deep Learning and SMC samplers can be parallelised by simultaneously considering parallelism across the data and the hypotheses. The studentship will then go on to undertake research that is only possible as a result of the existence of fast Bayesian Deep Learning (e.g., related to robust outlier detection).