A simple approach for quantizing neural networks
A simple approach for quantizing neural networks
复制标题
DOI：
10.48550/arxiv.2209.03487
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
J. Maly;Rayan Saab
中科院分区：
文献类型：
--
作者：
J. Maly;Rayan Saab
在这个简短的说明中，我们提出了一种新的方法来量化一个完全训练的神经网络的权重。一个简单的确定性预处理步骤允许我们通过无记忆标量量化来优化网络层，同时保持给定训练数据的网络性能。一方面，这种预处理的计算复杂度略高于文献中最先进的算法。另一方面，我们的方法不需要任何超参数调整，与以前的方法相比，允许简单的分析。我们在量化单个网络层的情况下提供了严格的理论保证，并表明如果训练数据表现良好，则相对误差随着网络中参数的数量而衰减，例如，如果它是从合适的随机分布中抽样的话。所开发的方法也很容易通过连续应用于单层来量化深度网络。
In this short note, we propose a new method for quantizing the weights of a fully trained neural network. A simple deterministic pre-processing step allows us to quantize network layers via memoryless scalar quantization while preserving the network performance on given training data. On one hand, the computational complexity of this pre-processing slightly exceeds that of state-of-the-art algorithms in the literature. On the other hand, our approach does not require any hyper-parameter tuning and, in contrast to previous methods, allows a plain analysis. We provide rigorous theoretical guarantees in the case of quantizing single network layers and show that the relative error decays with the number of parameters in the network if the training data behaves well, e.g., if it is sampled from suitable random distributions. The developed method also readily allows the quantization of deep networks by consecutive application to single layers.