BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization
BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization
复制标题
DOI：
--
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Huanrui Yang;Lin Duan;Yiran Chen;Hai Li
中科院分区：
文献类型：
--
作者：
Huanrui Yang;Lin Duan;Yiran Chen;Hai Li
混合精度量化可以在深度神经网络的性能和压缩比之间实现最优折衷，因此得到了广泛的研究。然而，它缺乏一种系统的方法来确定准确的量化方案。以前的方法要么只检查人工设计的小搜索空间，要么利用繁琐的神经结构搜索来探索广阔的搜索空间。这些方法不能有效地产生最优量化方案。本文从引入比特级稀疏性的新角度出发，提出了比特级稀疏量化(BSQ)来解决混合精度量化问题。我们将量化后的每一位权值视为一个独立的可训练变量，并引入了一种可微的位稀疏正则化。BSQ可以在一组权重元素上引入全零比特，实现动态精度降低，从而得到原始模型的混合精度量化方案。我们的方法能够使用单一的基于梯度的优化过程来探索全混合精度空间，只需要一个超参数来权衡性能和压缩。与以前的方法相比，BSQ在CIFAR-10和ImageNet数据集上的各种模型架构上实现了更高的精度和更高的比特减少。
Mixed-precision quantization can potentially achieve the optimal tradeoff between performance and compression rate of deep neural networks, and thus, have been widely investigated. However, it lacks a systematic method to determine the exact quantization scheme. Previous methods either examine only a small manually-designed search space or utilize a cumbersome neural architecture search to explore the vast search space. These approaches cannot lead to an optimal quantization scheme efficiently. This work proposes bit-level sparsity quantization (BSQ) to tackle the mixed-precision quantization from a new angle of inducing bit-level sparsity. We consider each bit of quantized weights as an independent trainable variable and introduce a differentiable bit-sparsity regularizer. BSQ can induce all-zero bits across a group of weight elements and realize the dynamic precision reduction, leading to a mixed-precision quantization scheme of the original model. Our method enables the exploration of the full mixed-precision space with a single gradient-based optimization process, with only one hyperparameter to tradeoff the performance and compression. BSQ achieves both higher accuracy and higher bit reduction on various model architectures on the CIFAR-10 and ImageNet datasets comparing to previous methods.