CRCNS US-Japan Research Proposal: Modeling the Dynamic Topological Representation of the Primate Visual System

这个项目的目标是通过建立计算机模型来了解我们是如何看待事物的。“很明显，我们学会了说话;不太明显的是，我们学会了看。婴儿的视力大约是20/400，这意味着他们在法律上是盲人，世界最初对他们来说看起来非常模糊。他们必须学会区分人（特别是他们的母亲和家人）以及玩具，食物和其他物体经过数月和数年的发展。我们的视力怎么会变得如此之好，以至于我们可以打球、读书和穿针？了解这是如何发生的一种方法是建立模拟大脑工作方式的计算模型。人工智能近年来随着深度神经网络的出现而蓬勃发展，深度神经网络是一种非常简化的大脑模型。它们能够识别人脸和物体，并能够创造自动驾驶汽车。然而，这些计算机视觉模型与我们自己的视觉系统之间存在根本差异，这使得它们不那么鲁棒。该项目将为这些模型添加更多人类视觉系统的功能。例如，我们有一个中央凹视网膜，它只能在视野的一个小斑点内实现高保真视觉，大约是手臂长度的拇指甲大小。因此，我们每秒钟移动眼睛大约3次，以使世界聚焦。这个项目将建立一个计算模型，它有一个中央凹的视网膜，“移动它的眼睛”，并考虑来自大脑记录的数据。最近的视觉系统模型已经以皮层记录为基准（CORnet，BrainScore），但似乎正在达到一个平台。为了超越这一点，下一代模型必须在解剖学和生理学上更接近大脑。该项目将结合卷积网络的根本变化以及来自灵长类视觉系统的新数据。视觉系统的大多数模型中缺少：1）生物学上真实的横向和反馈连接，包括具有全套横向相互作用的兴奋性（E）和抑制性（I）神经元的不同池（E-E，E-I，I-E，I-I），和纯兴奋性反馈连接; 2）从视网膜到V1的对数极坐标映射，将中央和周边表示分离，并增加旋转和尺度不变性;以及3）扫视，向表示添加动态。大多数神经生理学记录中缺少的是：1）在自由观察物体（扫视）期间来自IT的记录; 2）在从IT记录时对中枢和外周V1的药理学抑制，以测量它们对表征的贡献; 3）同时记录来自IT的多个区域，提供关于它们相互作用的关键数据。该项目将结合所有这些进展，以构建生物逼真的视觉系统。一个配套项目正在由日本国家信息和通信技术研究所（NICT）资助。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project is to understand how we see by building computer models that "see the way we do." It is obvious that we learn to talk; it is less obvious that we learn to see. Babies have roughly 20/400 vision, which means they are legally blind, and the world initially looks very blurry to them. They must learn to distinguish people (especially their mother and family) as well as toys, food, and other objects over months and years of development. How is it that we come to be able to see so well that we can play ball, read a book, and thread a needle? One way to understand how this happens is to build computational models that mimic the way the brain works. Artificial Intelligence has blossomed in recent years with the advent of deep neural networks, which are a very simplified model of the brain. They are capable of recognizing faces and objects, and are enabling the creation of self-driving cars. However, there are fundamental differences between these computer vision models and our own visual system that make them less robust. This project will add more features of the human visual system to these models. For example, we have a foveated retina, which enables high fidelity vision only within a small spot of the visual field, about the size of your thumbnail at arm's length. As a result, we move our eyes about 3 times a second in order to bring the world into focus. This project will build a computational model that has a foveated retina, "moves its eyes," and takes data from brain recordings into account.Recent models of the visual system have been benchmarked against cortical recordings (CORnet, BrainScore), but appear to be reaching a plateau. To move beyond this, the next generation of models will have to come closer to the brain in both anatomy and physiology. This project will incorporate radical changes to convolutional networks as well as novel data from the primate visual system. Missing from most models of the visual system are: 1) biologically realistic lateral and feedback connections, including distinct pools of excitatory (E) and inhibitory (I) neurons with the full set of lateral interactions (E-E, E-I, I-E, I-I), and purely excitatory feedback connections; 2) the log-polar mapping from retina to V1, separating central from peripheral representations and adding rotation and scale invariance; and 3) saccades, adding dynamics to the representations. Missing from most neurophysiological recordings are 1) recordings from IT during free viewing of objects (saccading); 2) pharmacological suppression of central and peripheral V1 while recording from IT in order to measure their contributions to representations; and 3) simultaneous recording from multiple areas of IT providing crucial data on their interactions. This project will incorporate all of these advances in order to build biologically realistic vision systems.A companion project is being funded by the National Institute of Information and Communications Technology, Japan (NICT).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.