Fast Bayesian Deep Learning

快速贝叶斯深度学习

• 批准号: 2297823
• 金额: --
• 依托单位: University of Liverpool
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2019
• 资助国家: 英国
• 起止时间: 2019 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2297823
• 关键词: Fast; Bayesian; Deep; Learning

Over recent years, Deep Learning has been demonstrated to offer impressive performance across a range of application domains. In part, this is because of its ability to exploit modern many-cored processors (e.g., GPUs). At its heart, Deep Learning makes use of stochastic gradient ascent, an approach that is known to struggle in the context of both local maxima and (the more numerous) saddle-points, degrading performance possible using a fixed dataset. Techniques have been developed that replace the stochastic gradient ascent with numerical Bayesian methods typified by Markov chain Monte Carlo (MCMC). Unfortunately, while such use of MCMC makes it theoretically possible to always find the global maximum, MCMC is so slow that this is hard to achieve in practice. This has restricted the extent to which MCMC has been used in the context of Deep Learning. Recent work by the University of Liverpool has developed a variant of the Sequential Monte Carlo (SMC) sampler, as a generic alternative to MCMC. SMC samplers appear to offer the same global optimisation capability as MCMC but with substantially reduced computational cost. This comes about, in part, because SMC samplers are inherently parallel, making it possible to exploit, for example, GPUs. Deep Learning achieves parallelism by distributing the processing across the data. SMC samplers achieve parallelism by distributing the consideration of the many hypotheses that the data could imply to be true. This studentship will begin by investigating the extent to which a combination of Deep Learning and SMC samplers can be parallelised by simultaneously considering parallelism across the data and the hypotheses. The studentship will then go on to undertake research that is only possible as a result of the existence of fast Bayesian Deep Learning (e.g., related to robust outlier detection).

近年来，深度学习已被证明在一系列应用领域提供令人印象深刻的性能。在某种程度上，这是因为它能够利用现代多核处理器（例如，GPU）。深度学习的核心是利用随机梯度上升，这种方法在局部最大值和（更多的）鞍点的背景下都很困难，使用固定数据集可能会降低性能。已经开发出用以马尔可夫链蒙特卡罗（MCMC）为代表的数值贝叶斯方法代替随机梯度上升的技术。不幸的是，虽然MCMC的这种使用在理论上可以总是找到全局最大值，但MCMC太慢，以至于在实践中很难实现。这限制了MCMC在深度学习背景下的使用程度。最近的工作由利物浦大学已经开发了一个变种的顺序蒙特卡罗（SMC）采样器，作为一个通用的替代MCMC。SMC采样器似乎提供了相同的全局优化能力MCMC，但大大降低了计算成本。这在一定程度上是因为SMC采样器本质上是并行的，因此可以利用GPU等。深度学习通过在数据中分布处理来实现并行性。SMC采样器通过分散考虑数据可能暗示为真的许多假设来实现并行性。该研究项目将开始，通过同时考虑数据和假设之间的并行性，研究深度学习和SMC采样器的组合可以并行化的程度。该奖学金将继续进行研究，这是唯一可能的快速贝叶斯深度学习的存在的结果（例如，与鲁棒离群值检测相关）。