NCS-FO: Collaborative Research: A Neurally-Inspired, Event-Based Computer Vision Pipeline

这项研究的目标是通过更紧密地模仿生物学，特别是利用事件驱动输入中存在的信息以及自上而下和横向反馈，使机器更加智能。尽管计算机继续进入日常生活，但它们仍然不能完成许多人类和其他动物容易完成的任务，或者远远落后于它们的生物同行。例如，在设计用于测量对象检测任务的性能的公开可用的数据集上，在最快的可用硬件上运行的最好的计算机软件无法定位20%-40%的人工注释目标。如果它的视觉系统犯了这么多错误，任何动物都不会长久存活，至少在野外是不会的。因此，这个项目将研究如何利用生物神经系统使用的信息处理策略来设计更强大的计算机硬件和软件。该项目将对计算机设计的神经启发方法方面的培训产生影响，并在国防和商业用途的神经形态计算方面发挥技术领先地位。该项目将研究在生物学中普遍存在但通常在计算机软件和硬件中找不到的处理机制。这些机制中的第一个是事件驱动输入。视网膜以离散脉冲的形式向大脑发送视觉信息，这些脉冲沿着视神经传播。同样，耳蜗会将声音编码为沿听神经传播的动作电位或棘波。在这两种情况下，给定神经纤维中出现尖峰的准确时间，相对于其他神经纤维中出现尖峰的时间，可以编码对大脑后续处理至关重要的信息。就视网膜而言，相对的棘波计时可能有助于将前景区域与背景区域分开，甚至有助于我们更快地阅读这一页上的单词。在耳蜗中，相对计时可以用来区分不同的声源，这反过来又让我们即使在嘈杂的房间里也能听到我们的同伴在说什么。相比之下，当今最常用的人工神经网络类型不使用事件驱动输入。这项研究将检验这样一个假设，即事件驱动的输入可以用于提高人工神经网络在图像和音频分割任务中的性能。第二个要研究的生物处理机制是自上而下的反馈。虽然今天研究的大多数人工神经网络都是严格的前馈网络，但大脑中的突触绝大多数来自自上而下和横向反馈连接。将使用一个数学上易于处理的自上而下和横向反馈模型来测试这样的假设，即这种连接可以用于提高人工神经网络在对象定位任务中的性能。还将探讨这种反馈是否能够降低网络对对抗性训练的敏感性。最后，该项目将探索一种根据环境输入自组织的新型神经形态芯片是否可以从事件驱动的输入中学习更强大的表示，并开发将此类芯片组合成具有横向和自上而下反馈的分层网络的策略。

The goal of this research is to make machines more intelligent by more closely mimicking biology, specifically, harnessing the information present in event-driven input along with top-down and lateral feedback. Although computers continue to make inroads into everyday life, they still cannot perform many tasks that are readily performed by humans and other animals, or lag far behind their biological counterparts. For example, on publically available datasets designed to measure performance on object detection tasks, the best computer software running on the fastest available hardware fails to locate between 20-40% of the human-annotated targets. No animal would long survive, at least not in the wild, if its visual system made so many errors. This project therefore will investigate how information processing strategies used by biological neural systems can be exploited to design more powerful computer hardware and software. The project will have impact on training in neurally-inspired approaches to computer design, and technological leadership in neuromorphic computing for both defense and commercial uses.The project will investigate processing mechanisms that are ubiquitous in biology but typically are not found in computer software and hardware. The first of these mechanisms is event-driven input. The retina sends visual information to the brain in the form of discrete pulses that propagate down the optic nerve. Likewise, the cochlea encodes sound as action potentials or spikes that propagate along the auditory nerve. In both cases, the precise time at which a spike occurs in a given nerve fiber, relative to the time at which spikes occur in other nerve fibers, can encode information that is critical to subsequent processing by the brain. In the case of the retina, relative spike timing may help to separate foreground from background regions or even help us to read more quickly the words on this page. In the cochlea, relative timing can be used to distinguish one sound source from another, which in turn allows us to hear what our companion is saying even while in a noisy room. In contrast, the types of artificial neural networks most commonly used today do not employ event-driven input. This research will test the hypothesis that event-driven input can be used to improve the performance of artificial neural networks on image and audio segmentation tasks. The second biological processing mechanism to be investigated is top-down feedback. Whereas most artificial neural networks studied today are strictly feed-forward, the vast preponderance of synapses in the brain arise from top-down and lateral feedback connections. A mathematically tractable model of top-down and lateral feedback will be used to test the hypothesis that such connections can be used to improve the performance of artificial neural networks on object localization tasks. Whether such feedback can reduce the susceptibility of networks to adversarial training will also be explored. Finally, the project will explore whether a new type of neuromorphic chip that self-organizes in response to environmental input can learn more powerful representation from event-driven input, and develop strategies for combining such chips into hierarchical networks with lateral and top-down feedback.