Neural networks learn highly selective representations in order to overcome the superposition catastrophe.
Neural networks learn highly selective representations in order to overcome the superposition catastrophe.
复制标题
神经网络学习高度选择性的表示以克服叠加灾难。
DOI：
--
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
C. Davis
中科院分区：
文献类型：
--
作者：
J. Bowers;Ivan I. Vankov;M. Damian;C. Davis
50年神经生理学的一个关键见解是，皮质中的一些神经元以高度选择性的方式对信息做出反应。为什么会这样呢？我们认为，选择性表征支持多个“事物”的共同激活（例如，单词、物体、面孔），而非选择性代码通常不适合于这一目的。也就是说，非选择性代码的共激活通常会导致模糊的混合模式，即所谓的叠加灾难。我们表明，一个经常性的并行分布式处理网络训练，在同一组单元的同时编码多个单词学习本地字母和单词代码，本地代码的数量与叠加的水平成比例。鉴于在短期记忆中，许多皮层系统需要共同激活多个事物，我们认为叠加约束在解释皮层中选择性代码的存在方面起着一定的作用。
A key insight from 50 years of neurophysiology is that some neurons in cortex respond to information in a highly selective manner. Why is this? We argue that selective representations support the coactivation of multiple "things" (e.g., words, objects, faces) in short-term memory, whereas nonselective codes are often unsuitable for this purpose. That is, the coactivation of nonselective codes often results in a blend pattern that is ambiguous; the so-called superposition catastrophe. We show that a recurrent parallel distributed processing network trained to code for multiple words at the same time over the same set of units learns localist letter and word codes, and the number of localist codes scales with the level of the superposition. Given that many cortical systems are required to coactivate multiple things in short-term memory, we suggest that the superposition constraint plays a role in explaining the existence of selective codes in cortex.