CRCNS US-Japan Research Proposal: Modeling the Dynamic Topological Representation of the Primate Visual System

这个项目的目标是通过建立“看我们做事的方式”的计算机模型来理解我们是如何看东西的。很明显，我们学会了说话；我们学会看的过程就不那么明显了。婴儿的视力大约是20/400，这意味着他们在法律上是失明的，世界最初对他们来说是非常模糊的。在数月或数年的发展过程中，他们必须学会区分人（尤其是他们的母亲和家人）以及玩具、食物和其他物体。我们的视力是如何变得如此之好，以至于我们可以打球、读书和穿针穿线？理解这种情况发生的一种方法是建立模拟大脑工作方式的计算模型。近年来，随着深度神经网络的出现，人工智能蓬勃发展，深度神经网络是一种非常简化的大脑模型。它们能够识别人脸和物体，并使自动驾驶汽车成为可能。然而，这些计算机视觉模型和我们自己的视觉系统之间存在着根本的差异，这使得它们不那么健壮。这个项目将为这些模型添加更多人类视觉系统的特征。例如，我们有一个注视点视网膜，它只能在视野的一小块区域内实现高保真视觉，大约是你的拇指指甲在一臂长时的大小。因此，我们的眼睛大约每秒移动3次，以使世界聚焦。这个项目将建立一个具有注视点视网膜的计算模型，“移动它的眼睛”，并将大脑记录的数据考虑在内。最近的视觉系统模型以皮质记录为基准（CORnet, BrainScore），但似乎正在达到一个平台。为了超越这一点，下一代模型必须在解剖学和生理学上更接近大脑。这个项目将结合卷积网络的根本变化以及灵长类视觉系统的新数据。大多数视觉系统模型中缺失的是：1)生物学上真实的横向和反馈连接，包括具有全套横向相互作用（E-E, E-I, I-E, I-I）的不同兴奋性(E)和抑制性(I)神经元池，以及纯粹的兴奋性反馈连接；2)从视网膜到V1的对数极坐标映射，将中心表示与外围表示分离，并添加旋转和比例不变性；3)扫视，增加动态表征。在大多数神经生理学记录中缺失的是：1)在自由观看物体时的IT记录（扫视）；2)中枢和外周V1的药理抑制，同时从IT记录，以衡量它们对表征的贡献；3)同时记录来自多个IT领域的信息，提供它们相互作用的关键数据。这个项目将结合所有这些进步，以建立生物学上逼真的视觉系统。日本国立信息通信技术研究所（NICT）正在资助一个伙伴项目。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project is to understand how we see by building computer models that "see the way we do." It is obvious that we learn to talk; it is less obvious that we learn to see. Babies have roughly 20/400 vision, which means they are legally blind, and the world initially looks very blurry to them. They must learn to distinguish people (especially their mother and family) as well as toys, food, and other objects over months and years of development. How is it that we come to be able to see so well that we can play ball, read a book, and thread a needle? One way to understand how this happens is to build computational models that mimic the way the brain works. Artificial Intelligence has blossomed in recent years with the advent of deep neural networks, which are a very simplified model of the brain. They are capable of recognizing faces and objects, and are enabling the creation of self-driving cars. However, there are fundamental differences between these computer vision models and our own visual system that make them less robust. This project will add more features of the human visual system to these models. For example, we have a foveated retina, which enables high fidelity vision only within a small spot of the visual field, about the size of your thumbnail at arm's length. As a result, we move our eyes about 3 times a second in order to bring the world into focus. This project will build a computational model that has a foveated retina, "moves its eyes," and takes data from brain recordings into account.Recent models of the visual system have been benchmarked against cortical recordings (CORnet, BrainScore), but appear to be reaching a plateau. To move beyond this, the next generation of models will have to come closer to the brain in both anatomy and physiology. This project will incorporate radical changes to convolutional networks as well as novel data from the primate visual system. Missing from most models of the visual system are: 1) biologically realistic lateral and feedback connections, including distinct pools of excitatory (E) and inhibitory (I) neurons with the full set of lateral interactions (E-E, E-I, I-E, I-I), and purely excitatory feedback connections; 2) the log-polar mapping from retina to V1, separating central from peripheral representations and adding rotation and scale invariance; and 3) saccades, adding dynamics to the representations. Missing from most neurophysiological recordings are 1) recordings from IT during free viewing of objects (saccading); 2) pharmacological suppression of central and peripheral V1 while recording from IT in order to measure their contributions to representations; and 3) simultaneous recording from multiple areas of IT providing crucial data on their interactions. This project will incorporate all of these advances in order to build biologically realistic vision systems.A companion project is being funded by the National Institute of Information and Communications Technology, Japan (NICT).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.