Self-Compression in Bayesian Neural Networks
Self-Compression in Bayesian Neural Networks
复制标题
DOI：
10.1109/mlsp49062.2020.9231550
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Giuseppina Carannante;Dimah Dera;G. Rasool;N. Bouaynaya
中科院分区：
文献类型：
--
作者：
Giuseppina Carannante;Dimah Dera;G. Rasool;N. Bouaynaya
机器学习模型已经在各种任务上实现了人类水平的性能。这一成功是以计算和存储开销为代价的，这使得机器学习算法难以在边缘设备上部署。通常，必须部分牺牲准确性，以有利于在减少内存使用和能耗方面量化的提高性能。目前的方法通过降低参数的精度或通过消除冗余参数来压缩网络。在本文中，我们提出了一个新的见解网络压缩通过贝叶斯框架。我们证明了贝叶斯神经网络可以自动发现模型参数中的冗余，从而实现自压缩，这与不确定性通过网络层的传播有关。我们的实验结果表明，网络架构可以成功地压缩，删除网络本身确定的参数，同时保持相同的精度水平。
Machine learning models have achieved human-level performance on various tasks. This success comes at a high cost of computation and storage overhead, which makes machine learning algorithms difficult to deploy on edge devices. Typically, one has to partially sacrifice accuracy in favor of an increased performance quantified in terms of reduced memory usage and energy consumption. Current methods compress the networks by reducing the precision of the parameters or by eliminating redundant ones. In this paper, we propose a new insight into network compression through the Bayesian framework. We show that Bayesian neural networks automatically discover redundancy in model parameters, thus enabling self-compression, which is linked to the propagation of uncertainty through the layers of the network. Our experimental results show that the network architecture can be successfully compressed by deleting parameters identified by the network itself while retaining the same level of accuracy.