EDGE - Adaptive Deep Learning Hardware for Embedded Platforms

深度学习是现代人工智能的关键技术，它在许多基于机器学习的应用中提供了最高的准确率。今天，虽然数字推理系统的大部分计算负荷仍然花在数据中心运行神经网络上，但随着智能手机的普及，以及即将推出的用于增强现实(AR)、虚拟现实(VR)和自主机器人系统的独立可穿戴设备的出现，随着数字逻辑技术的快速发展，对具有高能量和计算效率的数字推理硬件提出了更高的要求。最近，我们见证了DL体系结构类型的明显演变，提出了更复杂的网络体系结构来改进边缘AI推理。这包括以依赖数据的方式随每个新输入而变化的动态网络体系结构，其中输入和内部状态不是固定的。数字图书馆中的此类新架构概念可能会影响未来提供此类功能所需的硬件架构类型。该项目正好解决了这一挑战，并建议设计一种灵活的硬件体系结构，使其能够自适应地支持嵌入式设备上的各种DL算法。首先，为了生产成本更低、功耗更低、处理效率更高的DL-推理硬件，并且可以根据特定的应用规范和操作环境进行适应性配置，这将需要在现有DL技术的软件和硬件优化方面进行根本性的创新。这项工作旨在进行基础研究、开发和实用演示，以实现在资源和延迟预算有限的嵌入式边缘设备上对各种DL技术的普遍支持。首先，这需要在计算体系结构、存储层次和资源利用率以及系统延迟和吞吐量方面对现有的DL体系结构进行根本性的创新：对于现代DL系统来说，推理过程是动态的，例如DL推理可能是输入依赖的和资源依赖的。因此，该提案寻求以下三个方面：第一，在PI优化资源受限嵌入式设备的机器学习模型的现有工作的基础上，实现通过硬件感知近似技术根据需要动态优化网络模型的目标。其次，在可编程存储器层次结构和自适应处理硬件中新开发的自适应计算加速技术，寻求一个新的雄心勃勃的方向，开发一套上下文感知硬件体系结构，与能够充分利用真实硬件能力的近似算法密切合作。与传统的DL硬件推理引擎优化技术不同，该工作将探索自适应计算加速技术的软件和硬件可编程性，以最大化目标应用场景的优化结果。第三，该项目将与我们的行业和项目合作伙伴密切合作，制作一个实用的演示程序，展示拟议的DL框架相对于传统方法的有效性，特别是评估该框架在现实世界关键任务应用程序中的有效性。

Deep learning (DL) is the key technique in modern artificial intelligence (AI), which has provided state-of-the-art accuracy on many machine-learning based applications. Today, although most of the computational loads of DL systems are still spent running neural networks in data centres, the ubiquity of smartphones, and the upcoming availability of self-contained wearable devices for augmented reality (AR), virtual reality (VR) and autonomous robot systems are placing heavy demands on DL-inference hardware with high energy and computing efficiencies along with rapid development of DL techniques. Recently, we have witnessed a distinct evolution in the types of DL architecture, with more sophisticated network architectures proposed to improve edge AI inference. This includes dynamic network architectures that change with each new input in a data-dependent way, where inputs and internal states are not fixed. Such new architectural concepts in DL are likely to affect the type of hardware architectures that will be required to deliver such capabilities in the future. This project precisely addresses this challenge and proposes to design a flexible hardware architecture that enables adaptive support for a variety of DL algorithms on embedded devices. Primarily, to produce lower cost, lower power and higher processing efficiency DL-inference hardware that can be configured adaptably for dedicated application specifications and operating environments, this will require radical innovation in the optimisation of both the software and the hardware of current DL techniques.This work aims to perform fundamental research, development and practical demonstrator to enable general support for a variety of DL techniques on embedded edge devices with limited resource and latency budgets. Primarily, this requires radical innovation on the current DL architectures in terms of computing architecture, memory hierarchy and resource utilisation, as well as system latency and throughput: it is particularly important for the modern DL systems that the inference processes are dynamic, such as, the DL inference maybe input-dependent and resource-dependent. The proposal therefore seeks the following three thrusts: First, to build upon the existing work of the PI in optimising machine-learning models for resource-constrained embedded devices, towards achieving the goal that the network model could be dynamically optimised as needed through hardware-aware approximation techniques. Second, with newly-developed adaptive compute acceleration technology in programmable memory hierarchy and adaptive processing hardware, to seek a new ambitious direction to develop a set of context-aware hardware architectures to work closely with the approximation algorithms that can fully utilise the true hardware capabilities. Unlike traditional optimisation techniques for DL hardware inference engines, the proposed work will explore both software and hardware programmability of adaptive compute acceleration technology, in order to maximise the optimisation results for the target application scenarios. Third, this project will work closely with our industry and project partners to produce a practical demonstrator to showcase the effectiveness of the proposed DL framework versus traditional approaches, particularly, evaluating the effectiveness of the framework in real-world mission-critical applications.