Hebbian learning of context in recurrent neural networks
Hebbian learning of context in recurrent neural networks
复制标题
DOI：
10.1162/neco.1996.8.8.1677
复制
发表时间：
1996-11-15
影响因子：
2.9
通讯作者：
Brunel, N
中科院分区：
文献类型：
--
作者：
Brunel, N
在延迟的视觉记忆任务中，猴子颞叶下皮质的单电极记录为观察区域的吸引器动力学提供了证据。持续升高的延迟活动可能是训练期间向猴子展示的获得性视觉刺激特征的内部表征。当在训练过程中以固定的顺序呈现不相关的刺激时，这些实验在内部表征之间显示出显著的相关性。最近，一种简单的吸引子神经网络模型已经定量地再现了测量的相关性。该模型的一个基本假设是，在训练阶段期间形成的突触矩阵在其有效性中包含关于训练序列中持续刺激的邻接性的信息。我们在这里提出了一种简单的无监督学习动力学，如果刺激序列以固定的顺序重复呈现给网络，它就会产生这样的突触矩阵。然后将训练过程中的时间相关性转换为吸引子之间的空间相关性。场景是，在存在选择性延迟活动的情况下，在每个刺激的呈现时，神经组件中的活动分布包含当前刺激和前一个刺激(由吸引子携带)的信息。因此，递归突触矩阵不仅可以对呈现给网络的每个刺激进行编码，还可以对它们的上下文进行编码。我们结合了这样的想法，即为了学习有效，突触修改应该是随机的，以及吸引子提供关于两个连续刺激的可学习信息的事实。我们显式地计算了作为训练协议的函数的突触有效性的概率分布，即刺激呈现给网络的顺序。然后，我们求解由整合并激发的兴奋性和抑制性神经元组成的网络的动力学，该网络具有由学习动力学产生的突触侧支矩阵。网络有稳定的自发活动，稳定的延迟活动在关键的学习阶段后发展。学习动力学的可获得性使得对延迟活动分布的依赖性以及它们之间的相关性的一些实验预测成为可能，这些相关性取决于学习阶段和学习协议。特别是，它对配对关联延迟实验做出了具体的预测。
Single electrode recordings in the inferotemporal cortex of monkeys during delayed visual memory tasks provide evidence for attractor dynamics in the observed region. The persistent elevated delay activities could be internal representations of features of the learned visual stimuli shown to the monkey during training. When uncorrelated stimuli are presented during training in a fixed sequence, these experiments display significant correlations between the internal representations. Recently a simple model of attractor neural network has reproduced quantitatively the measured correlations. An underlying assumption of the model is that the synaptic matrix formed during the training phase contains in its efficacies information about the contiguity of persistent stimuli in the training sequence. We present here a simple unsupervised learning dynamics that produces such a synaptic matrix if sequences of stimuli are repeatedly presented to the network at fixed order. The resulting matrix is then shown to convert temporal correlations during training into spatial correlations between attractors. The scenario is that, in the presence of selective delay activity, at the presentation of each stimulus, the activity distribution in the neural assembly contains information of both the current stimulus and the previous one (carried by the attractor). Thus the recurrent synaptic matrix can code not only for each of the stimuli presented to the network but also for their context. We combine the idea that for learning to be effective, synaptic modification should be stochastic, with the fact that attractors provide learnable information about two consecutive stimuli. We calculate explicitly the probability distribution of synaptic efficacies as a function of training protocol, that is, the order in which stimuli are presented to the network. We then solve for the dynamics of a network composed of integrate-and-fire excitatory and inhibitory neurons with a matrix of synaptic collaterals resulting from the learning dynamics. The network has a stable spontaneous activity, and stable delay activity develops after a critical learning stage. The availability of a learning dynamics makes possible a number of experimental predictions for the dependence of the delay activity distributions and the correlations between them, on the learning stage and the learning protocol. In particular it makes specific predictions for pair-associates delay experiments.