Fast Bayesian Deep Learning

点击翻译按钮获取中文摘要

Over recent years, Deep Learning has been demonstrated to offer impressive performance across a range of application domains. In part, this is because of its ability to exploit modern many-cored processors (e.g., GPUs). At its heart, Deep Learning makes use of stochastic gradient ascent, an approach that is known to struggle in the context of both local maxima and (the more numerous) saddle-points, degrading performance possible using a fixed dataset. Techniques have been developed that replace the stochastic gradient ascent with numerical Bayesian methods typified by Markov chain Monte Carlo (MCMC). Unfortunately, while such use of MCMC makes it theoretically possible to always find the global maximum, MCMC is so slow that this is hard to achieve in practice. This has restricted the extent to which MCMC has been used in the context of Deep Learning. Recent work by the University of Liverpool has developed a variant of the Sequential Monte Carlo (SMC) sampler, as a generic alternative to MCMC. SMC samplers appear to offer the same global optimisation capability as MCMC but with substantially reduced computational cost. This comes about, in part, because SMC samplers are inherently parallel, making it possible to exploit, for example, GPUs. Deep Learning achieves parallelism by distributing the processing across the data. SMC samplers achieve parallelism by distributing the consideration of the many hypotheses that the data could imply to be true. This studentship will begin by investigating the extent to which a combination of Deep Learning and SMC samplers can be parallelised by simultaneously considering parallelism across the data and the hypotheses. The studentship will then go on to undertake research that is only possible as a result of the existence of fast Bayesian Deep Learning (e.g., related to robust outlier detection).