M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMs
M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMs
复制标题
DOI：
10.1109/icfpt59805.2023.00013
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Yuzong Chen;Jordan Dotzel;M. Abdelfattah
中科院分区：
文献类型：
--
作者：
Yuzong Chen;Jordan Dotzel;M. Abdelfattah
混合精度量化是压缩深度神经网络 (DNN) 的流行方法。然而，考虑到当前的 FPGA 架构和传统加速器数据流，利用混合精度 DNN 有效扩展性能具有挑战性。在这项工作中，我们通过提出 M4BRAM 来增强 FPGA 加速混合精度 DNN 的能力，M4BRAM 是一种新型的计算块 RAM (BRAM) 架构，可以计算混合精度矩阵-矩阵乘法。在精度方面，M4BRAM 支持多种混合精度 DNN 配置 - 权重精度可以为 2/4/8 位，而激活精度可以从 2 到 8 位不等。在数据流方面，M4BRAM 利用新颖的 BRAM 内数据复制方案来实现高硬件利用率。此外，在M4BRAM计算期间，其他FPGA资源可以无缝访问其数据，而不需要单独的缓冲区。因此，与之前的 BRAM 计算方案不同，M4BRAM 可以同时执行混合精度计算并保持作为存储器单元的全部功能，以真正补充 FPGA 上的现有计算资源。实验表明，将 M4BRAM 添加到平铺 DNN 加速器中，可以在 ImageNet 分类任务上的各种 DNN 上实现 $2.16\times$ 的平均加速，同时导致 < 0.5% 的准确率损失可以忽略不计。与采用先前的 BRAM 计算架构的相同平铺加速器相比，M4BRAM 在各种 DNN 中平均提供 1.43 美元\倍的性能提升。
Mixed-precision quantization is a popular approach for compressing deep neural networks (DNNs). However, it is challenging to scale the performance efficiently with mixed-precision DNNs given the current FPGA architecture and conventional accelerator dataflows. In this work, we enhance the FPGA’s capability for accelerating mixed-precision DNNs by proposing M4BRAM, a novel compute-in-block RAM (BRAM) architecture that can compute mixed-precision matrix-matrix multiplication. On the precision side, M4BRAM supports a wide range of mixed-precision DNN configurations – the weight precision can be 2/4/8 bits while the activation precision can vary from 2 to 8 bits. On the dataflow side, M4BRAM leverages a novel in-BRAM data duplication scheme to achieve high hardware utilization. Moreover, during M4BRAM computation, other FPGA resources can seamlessly access its data without the need for a separate buffer. Hence, unlike prior compute-in-BRAM proposals, M4BRAM can simultaneously perform mixed-precision computation and maintain full functionality as a memory unit to truly complement the existing compute resources on FPGAs. Experiments show that adding M4BRAM to a tiled DNN accelerator can achieve an average speedup of $2.16\times$ across various DNNs on the ImageNet classification task while incurring a negligible accuracy loss of < 0.5%. Compared to the same tiled accelerator that employs a prior compute-in-BRAM architecture, M4BRAM delivers $1.43\times$ higher performance on average across various DNNs.