Bitwise Neural Networks
Bitwise Neural Networks
复制标题
DOI：
--
复制
发表时间：
2016-01
期刊：
影响因子：
--
通讯作者：
Minje Kim;Paris Smaragdis
中科院分区：
文献类型：
--
作者：
Minje Kim;Paris Smaragdis
基于存在一个神经网络，有效地表示一组布尔函数之间的所有二进制输入和输出的假设，我们提出了一个开发和部署神经网络的过程，其权重参数，偏置项，输入和中间隐藏层输出信号，都是二进制值，只需要基本的位逻辑的前馈通过。所提出的按位神经网络（BNN）特别适合于资源受限的环境，因为它用更有效的按位运算取代了浮点运算或定点运算。因此，BNN需要更少的空间复杂度，更少的存储器带宽和更少的硬件功耗。为了设计这样的网络，我们建议添加一些训练方案，例如权重压缩和噪声反向传播，这会导致按位网络的性能几乎与其相应的实值网络一样好。我们在MNIST数据集上测试了所提出的网络，使用二进制特征表示，并表明BNN在提供显着计算节省的同时具有竞争力的性能。
Based on the assumption that there exists a neural network that efficiently represents a set of Boolean functions between all binary inputs and outputs, we propose a process for developing and deploying neural networks whose weight parameters, bias terms, input, and intermediate hidden layer output signals, are all binary-valued, and require only basic bit logic for the feedforward pass. The proposed Bitwise Neural Network (BNN) is especially suitable for resource-constrained environments, since it replaces either floating or fixed-point arithmetic with significantly more efficient bitwise operations. Hence, the BNN requires for less spatial complexity, less memory bandwidth, and less power consumption in hardware. In order to design such networks, we propose to add a few training schemes, such as weight compression and noisy backpropagation, which result in a bitwise network that performs almost as well as its corresponding real-valued network. We test the proposed network on the MNIST dataset, represented using binary features, and show that BNNs result in competitive performance while offering dramatic computational savings.