Fundamental aspects of noise in analog-hardware neural networks
Fundamental aspects of noise in analog-hardware neural networks
复制标题
DOI：
10.1063/1.5120824
复制
发表时间：
2019-10-01
期刊：
影响因子：
2.9
通讯作者：
Brunner, D.
中科院分区：
文献类型：
--
作者：
Semenova, N.;Porte, X.;Brunner, D.
我们研究和分析经常性以及深层多层网络中噪声传播的基本方面。我们研究的动机是模拟硬件中的神经网络;然而，该方法提供了对一般网络的深入了解。考虑到噪声线性节点，我们研究的信噪比在网络的输出，这决定了计算精度的上限。我们考虑加性和乘性噪声，这可以是纯粹的本地以及相关的神经元群体。这涵盖了硬件网络的主要内部扰动，并且噪声幅度从物理实现的神经网络获得。分析得出的描述与数值数据非常一致，从而能够清楚地识别对于管理和缓解噪音至关重要的组件。我们发现，模拟神经网络是惊人的强大，特别是对嘈杂的神经元。它们不相关的扰动几乎被完全抑制，而相关的噪声可以积累。我们的工作确定了众所周知的敏感点，同时突出了这种计算系统令人惊讶的鲁棒性。由AIP Publishing授权出版。
We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multilayer networks. The motivation of our study is neural networks in analog hardware; yet, the methodology provides insight into networks in general. Considering noisy linear nodes, we investigate the signal-to-noise ratio at the network's outputs, which determines the upper limit of computational precision. We consider additive and multiplicative noise, which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks, and noise amplitudes were obtained from a physically implemented neural network. Analytically derived descriptions agree exceptionally well with numerical data, enabling clear identification of the components critical for management and mitigation of noise. We find that analog neural networks are surprisingly robust, in particular, against noisy neurons. Their uncorrelated perturbations are almost fully suppressed, while correlated noise can accumulate. Our work identifies notoriously sensitive points while highlighting a surprising robustness of such computational systems. Published under license by AIP Publishing.