Hybrid no-propagation learning for multilayer neural networks
Hybrid no-propagation learning for multilayer neural networks
复制标题
DOI：
10.1016/j.neucom.2018.08.034
复制
发表时间：
2018-12-10
期刊：
影响因子：
6
通讯作者：
Kim, Hyongsuk
中科院分区：
文献类型：
--
作者：
Adhikari, Shyam Prasad;Yang, Changju;Kim, Hyongsuk
提出了一种适合于多层神经网络硬件实现的混合学习算法.虽然反向传播是多层神经网络的一种强大的学习方法，但由于神经突触的复杂性和误差反向传播中涉及的操作，其硬件实现是困难的。我们提出了一种学习算法，其性能与反向传播相当，但比反向传播更容易在硬件中实现，用于多层神经网络的片上学习。在所提出的学习算法中，一个多层神经网络的训练与基于梯度的增量规则和随机算法的混合，称为随机权重变化。输出层的参数使用delta规则学习，而内层参数使用随机权重变化学习，从而在不需要误差反向传播的情况下训练整个多层神经网络。实验结果表明，所提出的混合学习规则比其组成的学习算法，并与基准MNIST数据集上的反向传播的性能更好。硬件架构说明了易于实现的建议的学习规则，在模拟硬件相对于反向传播算法。(c)2018由Elsevier B.V.出版
A hybrid learning algorithm suitable for hardware implementation of multi- layer neural networks is proposed. Though backpropagation is a powerful learning method for multilayer neural networks, its hardware implementation is difficult due to complexities of the neural synapses and the operations involved in error backpropagation. We propose a learning algorithm with performance comparable to but easier than backpropagation to be implemented in hardware for on-chip learning of multi-layer neural networks. In the proposed learning algorithm, a multilayer neural network is trained with a hybrid of gradient- based delta rule and a stochastic algorithm, called Random Weight Change. The parameters of the output layer are learned using the delta rule, whereas the inner layer parameters are learned using Random Weight Change, thereby the overall multilayer neural network is trained without the need for error backpropagation. Experimental results showing better performance of the proposed hybrid learning rule than either of its constituent learning algorithms, and comparable to that of backpropagation on the benchmark MNIST dataset are presented. Hardware architecture illustrating the ease of implementation of the proposed learning rule in analog hardware vis-a-vis the backpropagation algorithm is also presented. (c) 2018 Published by Elsevier B.V.