Fast Bayesian Deep Learning

近年来，深度学习已被证明在一系列应用程序领域提供了令人印象深刻的性能。在一定程度上，这是因为它有能力开发现代多核处理器(例如，GPU)。深度学习的核心是利用随机梯度上升，这是一种已知在局部最大值和(更多的)鞍点的情况下都很难实现的方法，降低了使用固定数据集可能的性能。以马尔可夫链蒙特卡罗(MCMC)为代表的数值贝叶斯方法取代了随机梯度上升的技术。不幸的是，虽然这种MCMC的使用在理论上总是可以找到全局最大值，但MCMC速度太慢，以至于在实践中很难实现这一点。这限制了在深度学习背景下使用MCMC的程度。利物浦大学最近的工作开发了一种序列蒙特卡罗(SMC)采样器的变体，作为MCMC的通用替代品。SMC采样器似乎提供了与MCMC相同的全局优化能力，但计算成本大大降低。这在一定程度上是因为SMC采样器本质上是并行的，这使得利用例如GPU成为可能。深度学习通过跨数据分布处理来实现并行性。SMC采样器通过分散对数据可能暗示为真的许多假设的考虑来实现并行性。这项研究将通过同时考虑数据和假设的并行性，来调查深度学习和SMC采样器组合的并行度。然后，学生将继续进行只有在快速贝叶斯深度学习存在的情况下才可能进行的研究(例如，与稳健的离群点检测相关的研究)。

Over recent years, Deep Learning has been demonstrated to offer impressive performance across a range of application domains. In part, this is because of its ability to exploit modern many-cored processors (e.g., GPUs). At its heart, Deep Learning makes use of stochastic gradient ascent, an approach that is known to struggle in the context of both local maxima and (the more numerous) saddle-points, degrading performance possible using a fixed dataset. Techniques have been developed that replace the stochastic gradient ascent with numerical Bayesian methods typified by Markov chain Monte Carlo (MCMC). Unfortunately, while such use of MCMC makes it theoretically possible to always find the global maximum, MCMC is so slow that this is hard to achieve in practice. This has restricted the extent to which MCMC has been used in the context of Deep Learning. Recent work by the University of Liverpool has developed a variant of the Sequential Monte Carlo (SMC) sampler, as a generic alternative to MCMC. SMC samplers appear to offer the same global optimisation capability as MCMC but with substantially reduced computational cost. This comes about, in part, because SMC samplers are inherently parallel, making it possible to exploit, for example, GPUs. Deep Learning achieves parallelism by distributing the processing across the data. SMC samplers achieve parallelism by distributing the consideration of the many hypotheses that the data could imply to be true. This studentship will begin by investigating the extent to which a combination of Deep Learning and SMC samplers can be parallelised by simultaneously considering parallelism across the data and the hypotheses. The studentship will then go on to undertake research that is only possible as a result of the existence of fast Bayesian Deep Learning (e.g., related to robust outlier detection).