Noise-modulated neural networks for selectively functionalizing sub-networks by exploiting stochastic resonance
Noise-modulated neural networks for selectively functionalizing sub-networks by exploiting stochastic resonance
复制标题
DOI：
10.1016/j.neucom.2020.05.125
复制
发表时间：
2021-03
期刊：
影响因子：
6
通讯作者：
Shuhei Ikemoto
中科院分区：
文献类型：
--
作者：
Shuhei Ikemoto
在随机共振现象中，在非线性系统中加入一定程度的非零噪声可以减少信息损失。之前的一项研究提出了一种由阈值函数组成的神经网络，该函数在运行时和训练期间利用随机共振，目的是平滑映射和反向传播。这样的神经网络可以被重新表述为仅在添加噪声时操作的神经网络，即，当没有噪声时，不能平滑地映射和训练。同时着眼于这两种解释，本文提出了一种神经网络，其中只有一个子网络被激活的选择性地通过添加噪声的局部子网络。为此，引入了新的激活函数。它利用随机共振原理，在不加噪声的情况下，输出为零，导数为零。简单的模拟证实，建议的神经网络与新的激活功能，允许子网络被功能化选择性，和插值研究后，子网络分别训练后，通过对网络的各个区域施加不同的噪声强度。
In the phenomenon of stochastic resonance, adding a certain level of nonzero noise to a nonlinear system reduces information loss. A previous study proposed a neural network consisting of thresholding functions that exploit stochastic resonance at run time and during training, with the aim of smooth mapping and backpropagation. Such a neural network can be rephrased as one that operates only when noise is added, i.e., one that is unable to smoothly map and train when noise is absent. Focusing on both explanations simultaneously, a neural network for which only a sub-network is activated selectively by adding noise locally on that sub-network is proposed in this paper. To this end, a new activation function is introduced. It exploits stochastic resonance and presents null output and derivative when no noise is added. Simple simulations confirm that the proposed neural network with the new activation function allows the sub-network to be functionalized selectively, and interpolations are investigated by imposing varying noise intensity on various regions of the network after sub-networks are trained separately.