Quantization error-based regularization for hardware-aware neural network training
Quantization error-based regularization for hardware-aware neural network training
复制标题
DOI：
10.1587/nolta.9.453
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Kazutoshi Hirose;Ryota Uematsu;Kota Ando;Kodai Ueyoshi;M. Ikebe;T. Asai;M. Motomura;Shinya Takamaeda-Yamazaki
中科院分区：
文献类型：
--
作者：
Kazutoshi Hirose;Ryota Uematsu;Kota Ando;Kodai Ueyoshi;M. Ikebe;T. Asai;M. Motomura;Shinya Takamaeda-Yamazaki
提出了一种新的用于硬件感知神经网络训练的正则化策略“QER”。虽然量化的神经网络降低了计算能力和资源消耗，但由于数值表示的量化误差，它也降低了准确性，量化误差被定义为原始数字和量化数字之间的差异。QER通过将基于权重的量化误差的额外正则化项附加到损失函数来解决这样的问题。正则化项迫使权值的量化误差在减小原始损失的同时减小。我们评估我们的方法，使用MNIST一个简单的神经网络模型。评估结果表明，该方法实现了更高的精度比标准的训练方法与量化的前向传播。
: We propose “QER”, a novel regularization strategy for hardware-aware neural network training. Although quantized neural networks reduce computation power and resource consumption, it also degrades the accuracy due to quantization errors of the numerical representation, which are defined as differences between original numbers and quantized numbers. The QER solves such the problem by appending an additional regularization term based on quantization errors of weights to the loss function. The regularization term forces the quantization errors of weights to be reduced as well as the original loss. We evaluate our method by using MNIST on a simple neural network model. The evaluation results show that the proposed approach achieves higher accuracy than the standard training approach with quantized forward propagation.