A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems
A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems
复制标题
DOI：
10.1007/s12559-016-9389-5
复制
发表时间：
2016-10-01
影响因子：
5.4
通讯作者：
Karaoguz, Cem
中科院分区：
文献类型：
--
作者：
Gepperth, Alexander;Karaoguz, Cem
我们提出了一种受生物启发的增量学习体系结构，即使面对通常与感知问题相关的非常高的数据维度(>；1000)，该体系结构仍保持资源效率。特别是，我们调查了如何在不重新训练的情况下将新的感知(对象)类添加到训练的体系结构中，同时避免通常与此类场景相关的众所周知的灾难性遗忘效应。该建筑的核心是通过自组织的方法对感知空间进行生成性描述，同时将该空间中的邻域关系近似于二维平面上。这种近似紧密地模仿了视觉皮质的拓扑组织，即使在非常高维的情况下，也允许有效的局部更新规则来进行增量学习，我们在著名的MNIST基准测试中证明了这一点。我们通过增加一个生物学上看似合理的短期记忆系统来补充模型，使其即使在渐进学习的情况下也能保持出色的分类精度。短期存储器还被用来通过在专用的“睡眠”阶段期间重放先前存储的样本来加强新的数据统计。
We present a biologically inspired architecture for incremental learning that remains resource-efficient even in the face of very high data dimensionalities (> 1000) that are typically associated with perceptual problems. In particular, we investigate how a new perceptual (object) class can be added to a trained architecture without retraining, while avoiding the well-known catastrophic forgetting effects typically associated with such scenarios. At the heart of the presented architecture lies a generative description of the perceptual space by a self-organized approach which at the same time approximates the neighborhood relations in this space on a two-dimensional plane. This approximation, which closely imitates the topographic organization of the visual cortex, allows an efficient local update rule for incremental learning even in the face of very high dimensionalities, which we demonstrate by tests on the well-known MNIST benchmark. We complement the model by adding a biologically plausible short-term memory system, allowing it to retain excellent classification accuracy even under incremental learning in progress. The short-term memory is additionally used to reinforce new data statistics by replaying previously stored samples during dedicated "sleep" phases.