Defining mechanisms for natural vision in the primate brain with machine learning

通过机器学习定义灵长类动物大脑中的自然视觉机制

• 批准号: 10471557
• 负责人: CARLOS Ramon PONCE
• 金额: $ 152.55万
• 依托单位: HARVARD MEDICAL SCHOOL
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-30 至 2025-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10471557
• 关键词: Algorithms; Behavior; Bioprosthesis device; Brain; Categories; Chronic; Code; Complex; Data; Development; Educational workshop; Ethology; Experimental Designs; Face; Goals; Hour; Image; Implant; Knowledge; Laboratories; Lead; Macaca; Machine Learning; Medical Imaging; Methods; Microelectrodes; Modeling; Monkeys; Natural Selections; Nervous System Physiology; Neural Network Simulation; Neurons; Outcome; Pattern; Population; Primates; Problem Solving; Public Health; Research; Research Personnel; Semantics; Site; Stimulus; System; Terminology; Testing; Training; Vision; Visual; Visual Cortex; Visual impairment; Visual system structure; Work; awake; base; cortical visual impairment; design; generative adversarial network; inferotemporal cortex; neural network; nonhuman primate; object recognition; public health relevance; repository; response; sensory processing disorder; theories

PROJECT SUMMARY. To understand vision, we must be able to explain how it works in the natural world. Currently, while we can explain and predict how visual cortex neurons respond in limited laboratory conditions, most predictions fail when neurons are tested with randomly selected photographs of the natural world. One problem is that neurons in the monkey are usually tested using simple stimuli such as lines and dots, or using photographs from limited semantically defined categories (e.g. faces, places). In order to advance our knowledge of how neurons function with more complex visual inputs, we manipulated naturalistic images using neural network models called generative adversarial networks (GANs). GANS are trained to parameterize images similar to those in the natural world. We found that, when combined with evolutionary (search) algorithms, GANs synthesized images that highly activated inferotemporal cortex (IT) neurons, recorded using microelectrode arrays in awake behaving macaques. Neurons showed a range of response firing rates that exceed those elicited by previous approaches. The discovery of these highly activating images has energized a field-wide debate about how to best describe the tuning of neurons of the object-recognition system. Should neuronal activity be described using investigator-designed parametric frameworks (e.g., orientation, curvature), perceptual distances from highly activating images, or through data-fitted neural networks? In this proposed research, we will test these and other experimental designs to determine the best way to predict neuronal responses to natural images. We will pair approaches in a “tournament”-like meta-design, testing the same populations of neurons over hours and across days, using chronically implanted microelectrode arrays in awake, behaving non-human primates. We will also show which methods are best at predicting population response patterns comprising dozens of cortical visual sites. The project will include the development of workshops with other investigating teams in order to develop standard terminology and desiderata in explanatory theories of visual function. Although our own overarching theory is that the activity of a given cortical neuron represents the similarity from visual inputs to that neuron's most highly activating image (more precisely, to the visual attributes it contains), we will rely on neural networks as a unifying mechanism behind all approaches. The project will illustrate how to derive brain- wide organizational principles (based on ethology and concepts from natural selection) to explain visual recognition, and to constrain the space of neural networks that can best serve as models of the brain. The expected outcome is a framework for understanding how occipito-temporal neurons act in naturalistic image spaces, and how their representational capabilities inform recognition-based behaviors. Further, we will create code repositories (https://github.com/PonceLab) to encourage others to implement all approaches in their own research.

项目总结。要理解视觉，我们必须能够解释它在自然界中是如何工作的。 目前，虽然我们可以解释和预测视觉皮层神经元在有限的实验室条件下如何反应， 当用随机选择的自然界照片来测试神经元时，大多数预测都失败了。一 问题是，猴子的神经元通常是使用简单的刺激，如线和点，或使用 来自有限的语义定义的类别(例如，脸、地点)的照片。为了推进我们的 关于神经元如何在更复杂的视觉输入中发挥作用的知识，我们使用 神经网络模型称为生成性对抗网络(GANS)。Gans接受了参数化培训 与自然界相似的图像。我们发现，当与进化(搜索)相结合时 算法，Gans合成了高度激活下颞叶皮质(IT)神经元的图像，使用 清醒状态下猕猴的微电极阵列。神经元表现出一系列的反应放电率 超过了以前的方法所引发的。 这些高度活跃的图像的发现引发了一场关于如何最好地 描述物体识别系统的神经元的调节。神经元的活动是否应该用 调查员设计的参数框架(例如，方向、曲率)、与高度 激活图像，还是通过数据拟合的神经网络？在这项拟议的研究中，我们将测试这些和 其他实验设计，以确定预测神经元对自然图像反应的最佳方法。我们会 配对方法，在一个类似比赛的元设计中，在几个小时内测试相同的神经元群体 几天来，在清醒的灵长类动物中使用长期植入的微电极阵列，表现出非人类的行为。我们 还将展示哪些方法最适合预测包含数十个大脑皮层的种群反应模式 视觉站点。该项目将包括与其他调查小组共同举办讲习班，以便 开发视觉功能解释理论中的标准术语和期望数据。尽管我们自己 最重要的理论是，给定皮质神经元的活动代表了从视觉输入到 神经元最活跃的图像(更准确地说，是它所包含的视觉属性)，我们将依赖于 神经网络作为所有方法背后的统一机制。该项目将说明如何获得大脑- 广泛的组织原则(基于行为学和自然选择的概念)来解释视觉 识别，并限制最能作为大脑模型的神经网络的空间。 预期的结果是一个了解枕颞区神经元如何活动的框架。 自然主义图像空间，以及它们的表征能力如何影响基于识别的行为。 此外，我们将创建代码存储库(https://github.com/PonceLab)以鼓励其他人实现所有 在他们自己的研究中采用了一些方法。