Exploiting and Enhancing Programmable Logic for Deep Learning and Datacenter Acceleration

利用和增强可编程逻辑进行深度学习和数据中心加速

• 批准号: RGPIN-2022-04445
• 负责人: Betz, Vaughn
• 金额: $ 6.63万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=751564
• 关键词: Exploiting; Enhancing; Programmable; Logic; Deep

Computing has transformed society, enabling ubiquitous communications, on-demand entertainment, speech recognition, and more. While computation demand is growing for deep learning (DL) and other applications, the traditional efficiency gains afforded by transistor scaling are slowing down. To fill this gap, we need computational devices that are more efficient yet reprogrammable to allow new applications. Field-Programmable Gate Arrays (FPGAs) can be reprogrammed at the hardware level, enabling energy-efficiency gains of 10x or more vs. processors for many embedded and datacenter applications. We seek to advance on three related fronts: implementing efficient DL inference on FPGAs, architecting better reconfigurable devices and enhancing computer-aided design (CAD) tools to enable these new devices. Our first research thrust seeks more efficient DL inference on FPGAs while simultaneously creating productive development flows. In our heterogeneous pipeline (HPIPE) project, we leverage FPGA programmability by implementing customized hardware for every layer in a convolutional neural network (CNN) using a new domain specific compiler. Our neural processing unit (NPU) project instead creates DL functional units controlled by an instruction stream produced from software. Both projects have industry-leading performance, and we will enhance them in multiple ways, including scaling to multiple chips in parallel, exploiting the new tensor blocks in AI-optimized FPGAs, and combining the specialized units of HPIPE with the software programmability of the NPU. Our second thrust seeks new reconfigurable accelerator device (RAD) architectures to allow higher performance and easier development, particularly for DL and for datacenter infrastructure. We will investigate not only conventional 2D chips, but also the multi-die stacks enabled by recent technologies. We envision RADs that combine an FPGA fabric die on an infrastructure die containing coarse-grain programmable accelerators (such as hardened matrix-vector multiply units), large memory blocks, and an embedded network-on-chip (NoC) to link all the components. The combination of FPGA fabric and coarse-grained accelerators can increase performance, while the NoC decouples design components to simplify design. Our third thrust develops the computer-aided design (CAD) tools to investigate these RAD architectures and allow implementation of DL applications on them. First, we will develop a new tool (RADSim) to evaluate fabric, accelerator and NoC combinations by determining execution time for various applications on each architecture. Next, we will enhance the widely-used Versatile Place and Route (VPR) tool to co-optimize the placement of fabric resources, accelerator blocks and NoC routers, with latency and congestion estimates informed by RADsim. The open-source VPR tool is already enabling a wide variety of innovation and products, and these enhancements will make it still more capable.

计算改变了社会，使无处不在的通信、点播娱乐、语音识别等成为可能。虽然深度学习（DL）和其他应用的计算需求正在增长，但晶体管缩放带来的传统效率提升正在放缓。为了填补这一空白，我们需要更高效、可重新编程的计算设备，以允许新的应用。现场可编程门阵列（fpga）可以在硬件级别重新编程，与许多嵌入式和数据中心应用的处理器相比，可以实现10倍或更多的能效提升。我们寻求在三个相关方面取得进展：在fpga上实现高效的深度学习推理，构建更好的可重构设备，增强计算机辅助设计（CAD）工具以实现这些新设备。我们的第一个研究重点是寻求在fpga上更有效的深度学习推理，同时创造富有成效的开发流程。在我们的异构管道（HPIPE）项目中，我们利用FPGA可编程性，使用新的领域特定编译器为卷积神经网络（CNN）中的每一层实现定制硬件。我们的神经处理单元（NPU）项目创建了由软件产生的指令流控制的深度学习功能单元。这两个项目都具有行业领先的性能，我们将通过多种方式增强它们，包括并行扩展到多个芯片，利用ai优化fpga中的新张量块，以及将HPIPE的专用单元与NPU的软件可编程性相结合。我们的第二个重点是寻求新的可重构加速器设备（RAD）架构，以实现更高的性能和更容易的开发，特别是对于DL和数据中心基础设施。我们不仅将研究传统的2D芯片，还将研究由最新技术实现的多芯片堆栈。我们设想rad将FPGA结构芯片与包含粗粒度可编程加速器（如硬化矩阵向量乘法单元）、大内存块和嵌入式片上网络（NoC）的基础架构芯片结合起来，以连接所有组件。FPGA结构和粗粒度加速器的结合可以提高性能，而NoC解耦设计组件可以简化设计。我们的第三个重点是开发计算机辅助设计（CAD）工具来研究这些RAD架构并允许在其上实现DL应用程序。首先，我们将开发一个新工具（RADSim），通过确定每种架构上各种应用程序的执行时间来评估fabric、加速器和NoC组合。接下来，我们将增强广泛使用的多功能位置和路由（VPR）工具，以共同优化fabric资源、加速器块和NoC路由器的位置，并根据RADsim提供的延迟和拥塞估计进行评估。开源的VPR工具已经实现了各种各样的创新和产品，这些增强将使它更加强大。