ReLeQ: An Automatic Reinforcement Learning Approach for Deep Quantization of Neural Networks
ReLeQ: An Automatic Reinforcement Learning Approach for Deep Quantization of Neural Networks
复制标题
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Ahmed T. Elthakeb;Prannoy Pilligundla;FatemehSadat Mireshghallah;A. Yazdanbakhsh;H. Esmaeilzadeh
中科院分区：
文献类型：
--
作者：
Ahmed T. Elthakeb;Prannoy Pilligundla;FatemehSadat Mireshghallah;A. Yazdanbakhsh;H. Esmaeilzadeh
尽管深度神经网络（DNN）在广泛的现实任务中有许多最先进的应用，但两个主要挑战阻碍了DNN的进一步发展：超参数优化和受限的电源资源，这是嵌入式设备中的一个重要问题。由于计算强度和大内存占用，DNN随着规模的增长变得越来越难以训练和部署。最近的努力表明，通过减少内存带宽和使用有限的计算资源，将深度神经网络的权重量化为较低的位宽，朝着缓解上述问题迈出了重要的一步，这对于将DNN模型部署到资源有限的设备非常重要。本文建立在算法洞察的基础上，即DNN中操作的位宽可以在不影响其分类准确性的情况下减少。深度量化（量化位宽低于8），同时保持精度，需要大量的手动工作和超参数调整以及重新训练。本文通过设计一个端到端框架（称为ReLeQ）来解决上述问题，以自动化DNN量化。我们将DNN量化表示为优化问题，并使用最先进的基于策略梯度的强化学习（RL）算法，即邻近策略优化（PPO）来有效地探索DNN量化的大设计空间并解决定义的优化问题。为了证明ReLeQ的有效性，我们在几个神经网络上进行了评估，包括MNIST，CIFAR10，SVHN。ReLeQ将这些网络的权重量化为平均位宽分别为2.25、5和4，同时将最终的准确度损失保持在0.3%以下。
Despite numerous state-of-the-art applications of Deep Neural Networks (DNNs) in a wide range of real-world tasks, two major challenges hinder further advances in DNNs: hyperparameter optimization and constrained power resources, which is a significant concern in embedded devices. DNNs become increasingly difficult to train and deploy as they grow in size due to both computational intensity and the large memory footprint. Recent efforts show that quantizing weights of deep neural networks to lower bitwidths takes a significant step toward mitigating the mentioned issues, by reducing memory bandwidth and using limited computational resources which is important for deploying DNN models to devices with limited resources. This paper builds upon the algorithmic insight that the bitwidth of operations in DNNs can be reduced without compromising their classification accuracy. Deep quantization (quantizing bitwidths below eight) while maintaining accuracy, requires magnificent manual effort and hyper-parameter tuning as well as re-training. This paper tackles the aforementioned problems by designing an end to end framework, dubbed ReLeQ, to automate DNN quantization. We formulate DNN quantization as an optimization problem and use a state-of-the-art policy gradient based Reinforcement Learning (RL) algorithm, Proximal Policy Optimization (PPO) to efficiently explore the large design space of DNN quantization and solve the defined optimization problem. To show the effectiveness of ReLeQ, we evaluated it across several neural networks including MNIST, CIFAR10, SVHN. ReLeQ quantizes the weights of these networks to average bitwidths of 2.25, 5 and 4 respectively while maintaining the final accuracy loss below 0.3% .