How Many Bits Does it Take to Quantize Your Neural Network?
How Many Bits Does it Take to Quantize Your Neural Network?
复制标题
DOI：
10.1007/978-3-030-45237-7_5
复制
发表时间：
2020-03-13
期刊：
影响因子：
--
通讯作者：
Lechner M
中科院分区：
文献类型：
--
作者：
Giacobbe M;Henzinger TA;Lechner M
量化将神经网络转换为低位定点计算，可以由高效的仅整数硬件执行，并且是在实时嵌入式设备上部署神经网络的标准实践。然而，与它们的实数对应物一样，量化网络也不能免受对抗性攻击引起的恶意错误分类。我们研究了量化如何影响网络对对抗性攻击的鲁棒性，这是一个正式的验证问题。我们表明，无论是鲁棒性，也不是非鲁棒性是单调的，改变比特数的表示，也没有保留从一个实数网络的量化。出于这个原因，我们介绍了一种量化的神经网络的验证方法，使用SMT解决位向量，占其准确，位精确的语义。我们构建了一个工具，并分析了量化对MNIST数据集分类器的影响。我们证明，与我们的方法相比，现有的方法用于分析实数网络往往得出错误的结论，他们的量化，无论是在确定鲁棒性和检测攻击时，和现有的方法量化网络往往错过攻击。此外，我们将我们的方法应用于鲁棒性之外，展示了量化中的位数如何扩大学生成绩预测器的性别偏见。
Quantization converts neural networks into low-bit fixed-point computations which can be carried out by efficient integer-only hardware, and is standard practice for the deployment of neural networks on real-time embedded devices. However, like their real-numbered counterpart, quantized networks are not immune to malicious misclassification caused by adversarial attacks. We investigate how quantization affects a network’s robustness to adversarial attacks, which is a formal verification question. We show that neither robustness nor non-robustness are monotonic with changing the number of bits for the representation and, also, neither are preserved by quantization from a real-numbered network. For this reason, we introduce a verification method for quantized neural networks which, using SMT solving over bit-vectors, accounts for their exact, bit-precise semantics. We built a tool and analyzed the effect of quantization on a classifier for the MNIST dataset. We demonstrate that, compared to our method, existing methods for the analysis of real-numbered networks often derive false conclusions about their quantizations, both when determining robustness and when detecting attacks, and that existing methods for quantized networks often miss attacks. Furthermore, we applied our method beyond robustness, showing how the number of bits in quantization enlarges the gender bias of a predictor for students’ grades.
影响因子：
0.8
作者：
Pulina, Luca;Tacchella, Armando
通讯作者：
Tacchella, Armando
影响因子：
1.8
作者：
Singh, Gagandeep;Gehr, Timon;Vechev, Martin
通讯作者：
Vechev, Martin
DOI：
10.1109/tnnls.2018.2808470
发表时间：
2018-11-01
影响因子：
10.4
作者：
Xiang, Weiming;Hoang-Dung Tran;Johnson, Taylor T.
通讯作者：
Johnson, Taylor T.