Using Population Vectors to Understand Visual Working Memory for Natural Stimuli

使用群体向量来理解自然刺激的视觉工作记忆

• 批准号: 10339227
• 负责人: STEVEN J LUCK
• 金额: $ 39.56万
• 依托单位: UNIVERSITY OF CALIFORNIA AT DAVIS
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2025-12-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10339227
• 关键词: Algorithms; Architecture; Area; Attention; Behavior; Behavioral; Blinking; Brain region; Categories; Classification; Code; Color; Complex; Computer Models; Consensus; Data; Detection; Diagnosis; Disease; Electroencephalography; Event-Related Potentials; Eye Movements; Face; Frequencies; Functional Magnetic Resonance Imaging; Goals; Human; Individual Differences; Life; Literature; Location; Measures; Memory; Modeling; Nature; Neural Network Simulation; Neurons; Neurosciences; Noise; Pathway interactions; Pattern; Perception; Performance; Persons; Play; Population; Primates; Property; Proxy; Reporting; Research; Role; Sampling; Short-Term Memory; Specific qualifier value; Stimulus; Stream; Structure; Surface; System; Testing; To specify; Training; Vision; Visual; Visual Perception; Visual impairment; Vocabulary; Work; base; convolutional neural network; design; experimental study; feeding; human subject; insight; machine vision; neural patterning; object recognition; relating to nervous system; response; theories; vector; visual memory; visual processing; visual search

Visual working memory plays a fundamental role in visual perception and visually guided behavior, and much has been learned about the nature of this memory system by studies using arrays of artificial but easily controlled stimuli (e.g., arrays of colored disks or oriented Gabor patches). However, current quantitative models of visual working memory based on these artificial stimuli cannot be readily extended to the kinds of complex, structured scenes that humans face in daily life. The central goal of the proposed research is to develop and test a new quantitative approach to understanding the representation of complex objects and scenes in working memory, which will lead to a better understanding of real-world vision. In our model, a scene is represented in visual working memory as a noisier version of the pattern of activation that was produced during the perception of that scene. We model this by feeding the scene into a neural network model of the ventral object recognition pathway and using the resulting pattern of activation across the population of units (the population vector) as a model of the working memory representation. We then use this model to make predictions about both behavioral performance and neural activity in human subjects. For example, the change detection task involves presenting a sample scene followed after delay by a test scene and asking subjects to report whether the two scenes are the same or different. We can model this task by feeding the sample and test scenes into the model and computing the distance between the population vectors for the sample and test scenes. In our preliminary data, we find that the distance between the vectors can predict behavioral change detection performance extremely well. Moreover, we find that the vector for the sample scene can predict the pattern of neural activity during the delay between the sample and test scenes (measured via event-related potentials). Note that previous quantitative models of visual working memory cannot make any predictions at all for the natural scenes used in these preliminary studies. We propose testing and extending this model in several ways. First, we will conduct several experiments to assess the ability of the model to predict behavioral performance and neural activity across a broad range of natural stimuli. Second, we will compare the ability of population vectors from different cortical regions (as estimated from the model and from fMRI data) to predict behavioral performance and delay-period activity, providing new insights into the specific brain regions that underlie visual working memory. Third, we will determine whether our model can predict performance in visually guided tasks (e.g., visual search) that rely on visual working memory. Finally, we will assess different versions of our model that implement competing mechanisms for producing capacity limitations, and we will compare the ability of these models to account for behavioral performance for both natural scenes and classic examples of artificial stimuli. Together, these experiments will provide a new and broader understanding of visual working memory.

视觉工作记忆在视觉感知和视觉引导行为中起着重要作用， 通过使用人工但很容易的阵列进行研究，人们对这种记忆系统的本质有了很多了解， 受控刺激（例如，彩色圆盘阵列或定向的Gabor贴片）。然而，目前的定量 基于这些人工刺激的视觉工作记忆模型不能很容易地扩展到 人类在日常生活中所面对的复杂、结构化的场景。拟议研究的中心目标是 开发和测试一种新的定量方法来理解复杂对象的表示， 工作记忆中的场景，这将有助于更好地理解现实世界的视觉。 在我们的模型中，一个场景在视觉工作记忆中表现为一个更嘈杂的版本， 在感知那个场景时产生的激活。我们通过将场景输入到 腹侧物体识别通路的神经网络模型，并使用由此产生的激活模式 作为工作记忆表征的模型。我们 然后用这个模型来预测人类的行为表现和神经活动， 科目例如，变化检测任务涉及呈现样本场景，在延迟之后跟随有 测试场景，并要求受试者报告这两个场景是否相同或不同。我们可以模拟这个 通过将样本和测试场景馈送到模型中并计算总体之间的距离， 示例和测试场景的矢量。在我们的初步数据中，我们发现向量之间的距离 可以很好地预测行为变化检测性能。此外，我们发现， 样本场景可以预测样本场景和测试场景之间延迟期间的神经活动模式 （通过事件相关电位测量）。注意，先前的视觉工作记忆定量模型 我们无法对这些初步研究中使用的自然场景做出任何预测。 我们建议以几种方式测试和扩展这个模型。首先，我们将进行几个实验， 评估模型预测广泛范围内的行为表现和神经活动的能力 自然刺激。其次，我们将比较来自不同皮层区域的群体向量的能力（如 从模型和fMRI数据估计）来预测行为表现和延迟期活动， 为视觉工作记忆的特定大脑区域提供了新的见解。三是 确定我们的模型是否可以预测视觉引导任务中的性能（例如，视觉搜索）依赖于 视觉工作记忆最后，我们将评估我们的模型的不同版本， 生产能力限制的机制，我们将比较这些模型的能力，以说明 自然场景和人工刺激的经典例子的行为表现。所有这些 实验将提供一个新的和更广泛的理解视觉工作记忆。