NCS-FO: Collaborative Research: A Neurally-Inspired, Event-Based Computer Vision Pipeline

这项研究的目标是通过更紧密地模仿生物学，特别是利用事件驱动输入沿着的信息以及自上而下和横向反馈，使机器更加智能。 虽然计算机继续进入日常生活，但它们仍然不能执行许多人类和其他动物容易完成的任务，或者远远落后于生物同行。 例如，在设计用于测量目标检测任务性能的可用数据集上，在最快的硬件上运行的最好的计算机软件无法定位20-40%的人类注释目标。 如果动物的视觉系统出现如此多的错误，那么没有一种动物能长期存活，至少在野外是如此。 因此，本项目将研究如何利用生物神经系统使用的信息处理策略来设计更强大的计算机硬件和软件。 该项目将对神经启发的计算机设计方法的培训以及用于国防和商业用途的神经形态计算的技术领导产生影响。该项目将研究在生物学中普遍存在但通常在计算机软件和硬件中找不到的处理机制。 这些机制中的第一个是事件驱动输入。 视网膜将视觉信息以离散脉冲的形式发送到大脑，这些脉冲沿着视神经传播。 同样地，耳蜗将声音编码为动作电位或尖峰，其沿着听觉神经沿着传播。 在这两种情况下，相对于其他神经纤维中出现尖峰的时间，给定神经纤维中出现尖峰的精确时间可以编码对大脑后续处理至关重要的信息。 在视网膜的情况下，相对尖峰时间可能有助于将前景与背景区域分开，甚至有助于我们更快地阅读这一页上的单词。 在耳蜗中，相对定时可以用来区分一个声源与另一个声源，这反过来又使我们能够听到我们的同伴在说什么，即使在嘈杂的房间里。 相比之下，当今最常用的人工神经网络类型并不采用事件驱动输入。 这项研究将测试的假设，事件驱动的输入可以用来提高图像和音频分割任务的人工神经网络的性能。 第二个要研究的生物处理机制是自上而下的反馈。 尽管今天研究的大多数人工神经网络都是严格前馈的，但大脑中突触的巨大优势来自自上而下和横向反馈连接。 一个数学上易于处理的自上而下和横向反馈的模型将被用来测试的假设，这种连接可以用来提高人工神经网络的性能的对象定位任务。 这种反馈是否可以降低网络对对抗性训练的敏感性也将被探讨。 最后，该项目将探索一种新型的神经形态芯片，这种芯片可以根据环境输入进行自组织，是否可以从事件驱动的输入中学习更强大的表征，并开发将这种芯片结合到具有横向和自上而下反馈的分层网络中的策略。

The goal of this research is to make machines more intelligent by more closely mimicking biology, specifically, harnessing the information present in event-driven input along with top-down and lateral feedback. Although computers continue to make inroads into everyday life, they still cannot perform many tasks that are readily performed by humans and other animals, or lag far behind their biological counterparts. For example, on publically available datasets designed to measure performance on object detection tasks, the best computer software running on the fastest available hardware fails to locate between 20-40% of the human-annotated targets. No animal would long survive, at least not in the wild, if its visual system made so many errors. This project therefore will investigate how information processing strategies used by biological neural systems can be exploited to design more powerful computer hardware and software. The project will have impact on training in neurally-inspired approaches to computer design, and technological leadership in neuromorphic computing for both defense and commercial uses.The project will investigate processing mechanisms that are ubiquitous in biology but typically are not found in computer software and hardware. The first of these mechanisms is event-driven input. The retina sends visual information to the brain in the form of discrete pulses that propagate down the optic nerve. Likewise, the cochlea encodes sound as action potentials or spikes that propagate along the auditory nerve. In both cases, the precise time at which a spike occurs in a given nerve fiber, relative to the time at which spikes occur in other nerve fibers, can encode information that is critical to subsequent processing by the brain. In the case of the retina, relative spike timing may help to separate foreground from background regions or even help us to read more quickly the words on this page. In the cochlea, relative timing can be used to distinguish one sound source from another, which in turn allows us to hear what our companion is saying even while in a noisy room. In contrast, the types of artificial neural networks most commonly used today do not employ event-driven input. This research will test the hypothesis that event-driven input can be used to improve the performance of artificial neural networks on image and audio segmentation tasks. The second biological processing mechanism to be investigated is top-down feedback. Whereas most artificial neural networks studied today are strictly feed-forward, the vast preponderance of synapses in the brain arise from top-down and lateral feedback connections. A mathematically tractable model of top-down and lateral feedback will be used to test the hypothesis that such connections can be used to improve the performance of artificial neural networks on object localization tasks. Whether such feedback can reduce the susceptibility of networks to adversarial training will also be explored. Finally, the project will explore whether a new type of neuromorphic chip that self-organizes in response to environmental input can learn more powerful representation from event-driven input, and develop strategies for combining such chips into hierarchical networks with lateral and top-down feedback.