Variable Precision Multiplication for Software-Based Neural Networks
Variable Precision Multiplication for Software-Based Neural Networks
复制标题
DOI：
10.1109/hpec43674.2020.9286170
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Richa Singh;Thomas Conroy;P. Schaumont
中科院分区：
文献类型：
--
作者：
Richa Singh;Thomas Conroy;P. Schaumont
随着神经网络应用数量的不断增长，在高度受限的设备上高效执行推理计算的需求也在不断增长。在本文中，我们提出了一种加速软件中神经网络的方法。我们通过以位片计算格式制定重复操作来利用典型神经网络的有限精度要求。位片计算可确保 $M$ 位处理器字的每一位都贡献有用的工作，即使在计算有限精度 n 位（$n$ < M）运算时也是如此。本文带来以下贡献。我们首先提出一个环境，通过从 Verilog 综合它们，在软件中有效地创建位片描述。然后，我们开发矩阵乘法的位切片设计并评估其性能。我们的目标是小型微控制器，我们完全依赖软件优化。我们的驾驶应用程序是 MNIST 数据库的神经网络分类器。对称模式下的基于范围的线性量化可将预先训练的 32 位浮点权重和激活量化为低精度数据宽度。在具有不同硬件支持级别的 RISC-V 上进行的实验表明，对于神经网络应用程序常见的数据宽度，位切片代码比传统方法产生加速，从而实现更快、更高效的推理，而不会导致准确性的显着损失。例如，与没有硬件乘法器的非位片 rv32i ISA 实现相比，8 位矩阵乘法的速度提高了 2.62 倍。
As the number of applications of neural networks continues to grow, so does the need to efficiently perform inference computations on highly constrained devices. In this paper, we propose a methodology to accelerate neural networks in software. We exploit the limited-precision requirements of typical neural networks by formulating recurring operations in a bit-slice computation format. Bit-slice computation ensures that every bit of an $M$-bit processor word contributes useful work even while computing a limited-precision n-bit (with $n$ < M) operation. This paper brings the following contributions. We first present an environment to efficiently create bitslice descriptions in software, by synthesizing them from Verilog. We then develop bitsliced designs of matrix multiplication and evaluate their performance. Our target is a small microcontroller, and we rely solely on software optimization. Our driving application is a neural network classifier for the MNIST database. Range-Based Linear Quantization in symmetric mode quantizes pre-trained 32-bit floating point weights and activation to low-precision data-widths. Experiments on RISC-V with varying levels of hardware-support show that for data-widths common to neural network applications, the bit-sliced code produces a speedup over traditional methods, which leads to faster and efficient inference without incurring significant loss in accuracy. For example, 8-bit matrix multiplications are sped up by a factor of 2.62× when compared with non-bitsliced rv32i ISA implementation with no hardware multiplier.