Collaborative Research: Seer: A Gigascale Neuromorphic Visual System

该项目的目标是实时解析来自移动平台的视频，以生成显示场景空间布局以及存在的任何独立移动物体的视网膜定位图。该项目建议在一个多芯片神经形态系统中复制早期视觉系统的功能，该系统拥有23万个硅神经元（VGA图像中像素数量的四分之三）和30亿个突触相互作用，比迄今为止建造的任何东西都要大几个数量级。这个项目的成果将是SEER，一种亚瓦、平装本大小的视觉机器。所提出的研究的智力价值源于基于组合性原理的计算理论和基于可重入网络的神经形态实现之间的紧密协同。组合性决定了视觉问题的各个部分应该同时处理，而可重入性使我们能够做到这一点。拟议中的努力可能产生巨大的广泛影响。执行多模态分割的多芯片神经形态系统将重振机器人技术和计算机视觉。通过提供早期视觉的基础设施，它将促进认知的研究，这很可能会在神经科学和人工科学中产生大量新的理论和实验。

The goal of this project is to parse video from a moving platform in real-time to produce retinotopic maps that reveal the spatial layout of the scene as well as any independently moving objects present. The project proposes to duplicate the function of the early visual system in a multichip neuromorphic system with two hundred and thirty thousand silicon neurons (three quarters the number of pixels in a VGA image) and three billion synaptic interactions several orders of magnitude larger than anything built to date. The outcome of this project will be SEER, a subwatt, paperback-size seeing machine.The intellectual merit of the proposed research stems from the tight synergy between thecomputational theory, based on the principle of compositionality ,and the neuromorphic implementation, based on reentrant networks. Compositionality dictates that the various parts of the vision problem should be attacked simultaneously and reentrancy gives us the capability to do exactly that.The broader impact of the proposed effort could be enormous. A multichip neuromorphic system performing multimodal segmentations would reinvigorate robotics and computer vision. By providing the infrastructure for early vision, it will facilitate the study of cognition, which will most likely generate a flood of new theories and experiments, in the Neurosciences and in the sciences of the artificial.