Weight statistics controls dynamics in recurrent neural networks
Weight statistics controls dynamics in recurrent neural networks
复制标题
DOI：
10.1371/journal.pone.0214541
复制
发表时间：
2019-04-09
期刊：
影响因子：
3.7
通讯作者：
Metzner, Claus
中科院分区：
文献类型：
--
作者：
Krauss, Patrick;Schuster, Marc;Metzner, Claus
递归神经网络是复杂的非线性系统，能够在没有驱动输入的情况下持续活动。这些系统的动力学性质，特别是它们的长时间吸引子状态，在微观层面上由单个神经元之间的连接强度w（ij）决定。然而，很少有人知道，在何种程度上网络动态是可调的更粗粒度的水平上的统计特征的权重矩阵。在这项工作中，我们研究玻尔兹曼神经元的递归网络的动力学。特别是，我们研究了三个统计参数的影响：密度（非零连接的比例），平衡（兴奋性与抑制性连接的比例）和对称性（w（ij）= w（ij）的神经元对的比例）。通过计算网络动力学的“相图”，我们发现平衡是基本的控制参数：它从负值逐渐增加到正值，驱动系统从振荡行为进入混沌状态，并最终进入静止不动点。只有直接在混沌状态的边界处，神经网络才显示出丰富而规则的动态，从而实现实际的信息处理。这些结果表明，通过确保兴奋性和抑制性神经连接之间的适当平衡，大脑也被微调到“混乱的边缘”。
Recurrent neural networks are complex non-linear systems, capable of ongoing activity in the absence of driving inputs. The dynamical properties of these systems, in particular their long-time attractor states, are determined on the microscopic level by the connection strengths w(ij) between the individual neurons. However, little is known to which extent network dynamics is tunable on a more coarse-grained level by the statistical features of the weight matrix. In this work, we investigate the dynamics of recurrent networks of Boltzmann neurons. In particular we study the impact of three statistical parameters: density (the fraction of non-zero connections), balance (the ratio of excitatory to inhibitory connections), and symmetry (the fraction of neuron pairs with w(ij)= w(ij)). By computing a 'phase diagram' of network dynamics, we find that balance is the essential control parameter: Its gradual increase from negative to positive values drives the system from oscillatory behavior into a chaotic regime, and eventually into stationary fixed points. Only directly at the border of the chaotic regime do the neural networks display rich but regular dynamics, thus enabling actual information processing. These results suggest that the brain, too, is fine-tuned to the 'edge of chaos' by assuring a proper balance between excitatory and inhibitory neural connections.