A Gradient-Interleaved Scheduler for Energy-Efficient Backpropagation for Training Neural Networks
A Gradient-Interleaved Scheduler for Energy-Efficient Backpropagation for Training Neural Networks
复制标题
DOI：
10.1109/iscas45731.2020.9181242
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Nanda K. Unnikrishnan;K. Parhi
中科院分区：
文献类型：
--
作者：
Nanda K. Unnikrishnan;K. Parhi
本文讨论了加速器的设计，使用脉动架构的神经网络的训练，使用一种新的梯度交织方法。训练神经网络涉及误差的反向传播和关于激活函数和权重的梯度的计算。它示出，相对于激活函数的梯度可以使用一个重量固定的心脏收缩阵列计算，而相对于重量的梯度可以使用一个输出固定的心脏收缩阵列计算。所提出的方法的新奇在于交错的计算这两个梯度到同一个可配置的脉动阵列。这导致变量从一个计算到另一个计算的重用，并消除了不必要的内存访问。所提出的方法导致1.4-2.2倍节省的周期数和1.9倍节省的内存访问。因此，所提出的加速器减少了延迟和能量消耗。
This paper addresses design of accelerators using systolic architectures for training of neural networks using a novel gradient interleaving approach. Training the neural network involves backpropagation of error and computation of gradients with respect to the activation functions and weights. It is shown that the gradient with respect to the activation function can be computed using a weight-stationary systolic array while the gradient with respect to the weights can be computed using an output-stationary systolic array. The novelty of the proposed approach lies in interleaving the computations of these two gradients to the same configurable systolic array. This results in reuse of the variables from one computation to the other and eliminates unnecessary memory accesses. The proposed approach leads to 1.4–2.2× savings in terms of number of cycles and 1.9× savings in terms of memory accesses. Thus, the proposed accelerator reduces latency and energy consumption.