CRCNS US-Japan Research Proposal: Modeling the Dynamic Topological Representation of the Primate Visual System

这个项目的目标是通过构建“看到我们所做的方式”的计算机模型来了解我们是如何看待事物的。很明显，我们学会了说话；而我们学会了看，就不那么明显了。婴儿的视力大约为20/400，这意味着他们在法律上是盲人，最初对他们来说，世界看起来非常模糊。经过几个月和几年的发展，他们必须学会辨别人(特别是他们的母亲和家人)以及玩具、食物和其他物品。我们怎么能看得这么清楚，能够打球、读书和穿针呢？要理解这是如何发生的，一种方法是建立模拟大脑工作方式的计算模型。近年来，随着深度神经网络的出现，人工智能蓬勃发展，深度神经网络是一种非常简化的大脑模型。它们能够识别人脸和物体，并使自动驾驶汽车的创造成为可能。然而，这些计算机视觉模型和我们自己的视觉系统之间存在着根本的差异，这使得它们的健壮性变得不那么强。该项目将为这些模型添加更多人类视觉系统的功能。例如，我们有一个凹陷的视网膜，它只能在视野中的一个小点上实现高保真视觉，大约与你的缩略图大小相当。因此，我们每秒大约移动3次眼睛，以使世界成为焦点。该项目将建立一个计算模型，它有一个中心凹的视网膜，“移动它的眼睛”，并考虑到来自大脑记录的数据。最近的视觉系统模型已经与皮质记录(Cornet，BrainScore)进行了基准比较，但似乎正达到一个平台期。为了超越这一点，下一代模型将不得不在解剖学和生理学上更接近大脑。该项目将结合卷积网络的根本变化以及来自灵长类视觉系统的新数据。大多数视觉系统模型缺少：1)生物真实的侧向和反馈连接，包括不同的兴奋性(E)和抑制性(I)神经元与全套侧向相互作用(E-E，E-I，I-E，I-I)的池，以及纯粹的兴奋性反馈连接；2)从视网膜到V1的对数极映射，将中央表示与外围表示分开，并增加旋转和比例不变性；以及3)扫视，增加表示的动力学。大多数神经生理学记录中缺少1)自由观看物体时来自IT的记录(扫视)；2)从IT记录时对中枢和外周V1的药物抑制，以测量它们对表征的贡献；以及3)从IT的多个区域同时记录，提供关于它们相互作用的关键数据。该项目将包括所有这些进展，以建立生物逼真的视觉系统。日本国家信息和通信技术研究所(NICT)正在资助一个配套项目。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project is to understand how we see by building computer models that "see the way we do." It is obvious that we learn to talk; it is less obvious that we learn to see. Babies have roughly 20/400 vision, which means they are legally blind, and the world initially looks very blurry to them. They must learn to distinguish people (especially their mother and family) as well as toys, food, and other objects over months and years of development. How is it that we come to be able to see so well that we can play ball, read a book, and thread a needle? One way to understand how this happens is to build computational models that mimic the way the brain works. Artificial Intelligence has blossomed in recent years with the advent of deep neural networks, which are a very simplified model of the brain. They are capable of recognizing faces and objects, and are enabling the creation of self-driving cars. However, there are fundamental differences between these computer vision models and our own visual system that make them less robust. This project will add more features of the human visual system to these models. For example, we have a foveated retina, which enables high fidelity vision only within a small spot of the visual field, about the size of your thumbnail at arm's length. As a result, we move our eyes about 3 times a second in order to bring the world into focus. This project will build a computational model that has a foveated retina, "moves its eyes," and takes data from brain recordings into account.Recent models of the visual system have been benchmarked against cortical recordings (CORnet, BrainScore), but appear to be reaching a plateau. To move beyond this, the next generation of models will have to come closer to the brain in both anatomy and physiology. This project will incorporate radical changes to convolutional networks as well as novel data from the primate visual system. Missing from most models of the visual system are: 1) biologically realistic lateral and feedback connections, including distinct pools of excitatory (E) and inhibitory (I) neurons with the full set of lateral interactions (E-E, E-I, I-E, I-I), and purely excitatory feedback connections; 2) the log-polar mapping from retina to V1, separating central from peripheral representations and adding rotation and scale invariance; and 3) saccades, adding dynamics to the representations. Missing from most neurophysiological recordings are 1) recordings from IT during free viewing of objects (saccading); 2) pharmacological suppression of central and peripheral V1 while recording from IT in order to measure their contributions to representations; and 3) simultaneous recording from multiple areas of IT providing crucial data on their interactions. This project will incorporate all of these advances in order to build biologically realistic vision systems.A companion project is being funded by the National Institute of Information and Communications Technology, Japan (NICT).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.