Automatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators
Automatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators
复制标题
DOI：
10.1109/tcad.2018.2884972
复制
发表时间：
2020-02-01
影响因子：
2.9
通讯作者：
Seo, Jae-sun
中科院分区：
文献类型：
--
作者：
Ma, Yufei;Cao, Yu;Seo, Jae-sun
卷积神经网络（convolutional neural networks, CNNs）的快速发展使其越来越广泛的应用。基于FPGA的CNN推理加速器由于其高性能和低功耗以及FPGA的可重构性和灵活性的传统优势而越来越受欢迎。然而，如果没有通用编译器来自动化实现，仍然需要大量的努力和专业知识来定制每个CNN模型的设计。在本文中，我们提出了一种寄存器传输级（RTL）级CNN编译器，该编译器可以自动为各种CNN的推理任务生成定制的FPGA硬件，以实现从软件到FPGA的CNN高级快速原型设计，同时仍然保持低级硬件优化的好处。首先，开发了一个通用的RTL模块库，对每一层的不同操作进行建模。物理模块的集成和数据流在顶层系统模板中预先定义，并在编译时对给定的CNN算法进行重新配置。逐层顺序计算的运行时控制由提出的执行计划管理，因此即使是高度不规则和复杂的网络拓扑，如GoogLeNet和ResNet，也可以编译。提出的方法在两个独立的英特尔fpga Arria 10和Stratix 10上用各种CNN算法（例如NiN， VGG， GoogLeNet和ResNet）进行了演示，分别实现了969 GOPS和1604 GOPS的端到端推理吞吐量，批大小为1。
A broad range of applications are increasingly benefiting from the rapid and flourishing development of convolutional neural networks (CNNs). The FPGA-based CNN inference accelerator is gaining popularity due to its high-performance and low-power as well as FPGA's conventional advantage of reconfigurability and flexibility. Without a general compiler to automate the implementation, however, significant efforts and expertise are still required to customize the design for each CNN model. In this paper, we present an register-transfer level (RTL)-level CNN compiler that automatically generates customized FPGA hardware for the inference tasks of various CNNs, in order to enable high-level fast prototyping of CNNs from software to FPGA and still keep the benefits of low-level hardware optimization. First, a general-purpose library of RTL modules is developed to model different operations at each layer. The integration and dataflow of physical modules are predefined in the top-level system template and reconfigured during compilation for a given CNN algorithm. The runtime control of layer-by-layer sequential computation is managed by the proposed execution schedule so that even highly irregular and complex network topology, e.g., GoogLeNet and ResNet, can be compiled. The proposed methodology is demonstrated with various CNN algorithms, e.g., NiN, VGG, GoogLeNet, and ResNet, on two standalone Intel FPGAs, Arria 10, and Stratix 10, achieving end-to-end inference throughputs of 969 GOPS and 1604 GOPS, respectively, with batch size of one.