Optimisation of Probabilistic Deep Learning Approaches for Hardware Acceleration

用于硬件加速的概率深度学习方法的优化

• 批准号: 2621264
• 金额: --
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2621264
• 关键词: Optimisation; Probabilistic; Deep; Learning; Approaches

The focus of this project is to investigate and develop methodologies and approximation schemes for the efficient mapping of probabilistic models in deep learning, such as Bayesian Neural Networks, on low-power embedded devices such as CPUs (Central Processing Units), NPUs (Neural Processing Units), and FPGAs (Field Programmable Gate Arrays). Despite the significant progress of neural network acceleration, it is well known that conventional neural networks can be prone to overfitting and poor generalisation - where the model fails to generalise well from the training data to test (unseen) data. It is often the case that typical deep neural network models do not provide reliable estimates of uncertainty alongside their predictions - they can be overconfident on unseen data. Robust uncertainties are important for real-life scenarios - e.g., autonomous driving, healthcare etc., where decisions may be made using uncertainty metrics as a basis. There exists a plethora of research, often under a probabilistic framework, with the aim of tackling this problem in deep learning, such as Bayesian Neural Networks and Deep Ensembles. However, these approaches are often computationally cumbersome. The goal of this research is to improve our understanding of the computational aspects of these types of models and to explore various optimisation strategies across the full stack (from model to hardware) for efficient implementation of them on low-power embedded platforms - e.g. CPU, NPU etc.Area: Artificial intelligence technologies

该项目的重点是研究和开发在低功耗嵌入式设备(如cpu（中央处理单元）、npu（神经处理单元）和fpga（现场可编程门阵列）上有效映射深度学习中概率模型（如贝叶斯神经网络）的方法和近似方案。尽管神经网络在加速方面取得了重大进展，但众所周知，传统的神经网络容易出现过拟合和泛化不良的问题——模型不能很好地从训练数据泛化到测试（看不见的）数据。通常情况下，典型的深度神经网络模型不能在预测的同时提供对不确定性的可靠估计——它们可能对看不见的数据过于自信。强大的不确定性对于现实生活场景非常重要，例如，自动驾驶、医疗保健等，在这些场景中，可能会使用不确定性度量作为基础做出决策。有大量的研究，通常在概率框架下，旨在解决深度学习中的这个问题，如贝叶斯神经网络和深度集成。然而，这些方法通常在计算上很麻烦。本研究的目标是提高我们对这些类型模型的计算方面的理解，并探索跨全栈（从模型到硬件）的各种优化策略，以便在低功耗嵌入式平台上有效实现它们-例如CPU， NPU等。领域：人工智能技术