Using Population Vectors to Understand Visual Working Memory for Natural Stimuli

使用群体向量来理解自然刺激的视觉工作记忆

• 批准号: 10543101
• 负责人: STEVEN J LUCK
• 金额: $ 39.94万
• 依托单位: UNIVERSITY OF CALIFORNIA AT DAVIS
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2025-12-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10543101
• 关键词: Algorithms; Architecture; Area; Attention; Behavior; Behavioral; Blinking; Brain region; Categories; Classification; Code; Color; Complex; Computer Models; Consensus; Data; Detection; Diagnosis; Disease; Electroencephalography; Event-Related Potentials; Eye Movements; Face; Frequencies; Functional Magnetic Resonance Imaging; Goals; Human; Individual Differences; Learning; Life; Literature; Location; Measures; Memory; Modeling; Nature; Neural Network Simulation; Neurons; Neurosciences; Noise; Pathway interactions; Pattern; Perception; Performance; Persons; Play; Population; Primates; Property; Proxy; Reporting; Research; Role; Sampling; Short-Term Memory; Specific qualifier value; Stimulus; Stream; Structure; Surface; System; Testing; Training; Vision; Visual; Visual Perception; Visual impairment; Vocabulary; Work; base; behavior prediction; convolutional neural network; design; experimental study; feeding; human subject; insight; machine vision; neural; neural patterning; object recognition; response; theories; vector; verbal; visual memory; visual processing; visual search

Visual working memory plays a fundamental role in visual perception and visually guided behavior, and much has been learned about the nature of this memory system by studies using arrays of artificial but easily controlled stimuli (e.g., arrays of colored disks or oriented Gabor patches). However, current quantitative models of visual working memory based on these artificial stimuli cannot be readily extended to the kinds of complex, structured scenes that humans face in daily life. The central goal of the proposed research is to develop and test a new quantitative approach to understanding the representation of complex objects and scenes in working memory, which will lead to a better understanding of real-world vision. In our model, a scene is represented in visual working memory as a noisier version of the pattern of activation that was produced during the perception of that scene. We model this by feeding the scene into a neural network model of the ventral object recognition pathway and using the resulting pattern of activation across the population of units (the population vector) as a model of the working memory representation. We then use this model to make predictions about both behavioral performance and neural activity in human subjects. For example, the change detection task involves presenting a sample scene followed after delay by a test scene and asking subjects to report whether the two scenes are the same or different. We can model this task by feeding the sample and test scenes into the model and computing the distance between the population vectors for the sample and test scenes. In our preliminary data, we find that the distance between the vectors can predict behavioral change detection performance extremely well. Moreover, we find that the vector for the sample scene can predict the pattern of neural activity during the delay between the sample and test scenes (measured via event-related potentials). Note that previous quantitative models of visual working memory cannot make any predictions at all for the natural scenes used in these preliminary studies. We propose testing and extending this model in several ways. First, we will conduct several experiments to assess the ability of the model to predict behavioral performance and neural activity across a broad range of natural stimuli. Second, we will compare the ability of population vectors from different cortical regions (as estimated from the model and from fMRI data) to predict behavioral performance and delay-period activity, providing new insights into the specific brain regions that underlie visual working memory. Third, we will determine whether our model can predict performance in visually guided tasks (e.g., visual search) that rely on visual working memory. Finally, we will assess different versions of our model that implement competing mechanisms for producing capacity limitations, and we will compare the ability of these models to account for behavioral performance for both natural scenes and classic examples of artificial stimuli. Together, these experiments will provide a new and broader understanding of visual working memory.

视觉工作记忆在视觉感知和视觉引导行为中起着重要作用，并且 通过使用人工但容易的阵列进行的研究，人们已经对这种记忆系统的性质有了很多了解 受控刺激(例如，彩色圆盘阵列或定向Gabor贴片)。然而，目前的量化 基于这些人工刺激的视觉工作记忆模型不容易扩展到 人类在日常生活中面临的复杂、结构化的场景。拟议研究的中心目标是 开发和测试一种新的量化方法来理解复杂对象的表示和 工作记忆中的场景，这将导致对现实世界视觉的更好理解。 在我们的模型中，场景在视觉工作记忆中被表示为 在感知那个场景的过程中产生的激活。我们通过将场景馈送到 腹侧物体识别路径的神经网络模型和使用所产生的激活模式 作为工作记忆表征的模型的跨单元种群(种群矢量)。我们 然后使用该模型对人类的行为表现和神经活动进行预测 研究对象。例如，变化检测任务涉及在延迟之后呈现一个样例场景 测试场景，并要求受试者报告两个场景是否相同或不同。我们可以对此进行建模 通过将样本和测试场景输入模型并计算种群之间的距离来完成任务 样例和测试场景的向量。在我们的初步数据中，我们发现向量之间的距离 可以非常好地预测行为变化检测性能。此外，我们还发现，向量对于 样本场景可以预测样本和测试场景之间的延迟期间的神经活动模式 (通过事件相关电位测量)。请注意，以前的视觉工作记忆量化模型 对于这些初步研究中使用的自然场景，根本无法做出任何预测。 我们建议通过几种方式测试和扩展该模型。首先，我们将进行几个实验，以 评估该模型在广泛范围内预测行为表现和神经活动的能力 自然刺激。第二，我们将比较来自不同大脑皮层区域(AS)的种群载体的能力 从模型和fMRI数据估计)以预测行为性能和延迟期活动， 提供了对构成视觉工作记忆的特定大脑区域的新见解。第三，我们将 确定我们的模型是否可以预测视觉引导任务(例如，视觉搜索)中的性能 视觉工作记忆。最后，我们将评估实现竞争的模型的不同版本 生产能力限制的机制，我们将比较这些模型的解释能力 自然场景和人工刺激的经典例子的行为表现。加在一起，这些 实验将提供对视觉工作记忆的新的、更广泛的理解。