AxTrain: Hardware-Oriented Neural Network Training for Approximate Inference
AxTrain: Hardware-Oriented Neural Network Training for Approximate Inference
复制标题
DOI：
10.1145/3218603.3218643
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Xin He;Liu Ke;Wenyan Lu;Guihai Yan;Xuan Zhang
中科院分区：
文献类型：
--
作者：
Xin He;Liu Ke;Wenyan Lu;Guihai Yan;Xuan Zhang
神经网络固有的容错性使得近似计算成为提高神经网络推理能量效率的一种很有前途的技术。传统的近似计算侧重于权衡现有预先训练网络的效率和精度之间的权衡，这可能会导致次优解。在本文中，我们提出了一个面向硬件的训练框架AxTrain，以方便神经网络推理的近似计算。具体地说，AxTrain利用了两种正交方法之间的协同作用-一种是主动搜索具有高容错性的网络参数分布，另一种是被动地通过在训练阶段结合前向传递中近似硬件的噪声分布来学习弹性权重。在采用近阈值计算和近似乘法策略的各种数据集上的实验结果表明，AxTrain能够获得弹性神经网络参数，并提高系统的能量效率。
The intrinsic error tolerance of neural network (NN) makes approximate computing a promising technique to improve the energy efficiency of NN inference. Conventional approximate computing focuses on balancing the efficiency-accuracy trade-off for existing pre-trained networks, which can lead to suboptimal solutions. In this paper, we propose AxTrain, a hardware-oriented training framework to facilitate approximate computing for NN inference. Specifically, AxTrain leverages the synergy between two orthogonal methods---one actively searches for a network parameters distribution with high error tolerance, and the other passively learns resilient weights by numerically incorporating the noise distributions of the approximate hardware in the forward pass during the training phase. Experimental results from various datasets with near-threshold computing and approximation multiplication strategies demonstrate AxTrain's ability to obtain resilient neural network parameters and system energy efficiency improvement.