Visual Perception as Retrospective Bayesian Decoding from High- to Low-level Features in Working Memory

视觉感知作为从工作记忆中的高级到低级特征的回顾性贝叶斯解码

• 批准号: 1754211
• 负责人: Ning Qian
• 金额: $ 51.32万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-04-01 至 2022-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1754211&HistoricalAwards=false
• 关键词: Visual; Perception; Retrospective; Bayesian; Decoding

When looking at a scene, we typically have a quick and accurate perceptual understanding of its high-level category, for example, a home, office, street, or jungle. We rarely pay attention to the scene's low-level properties such as luminance values at various spots unless we are asked to report them, and even then, we are not very accurate about them. The fact that higher-level properties of a scene are more relevant to our behavior than low-level properties has informed global precedence theories of perception. However, experimental studies of the brain have established that lower-level features in a scene are detected before higher-level features; this result somehow led to the commonly used, but rarely tested, assumption that visual perception follows the same low-to-high-level hierarchy of feature detection. This project attempts to resolve this apparent contradiction by separating feature detection and perception, and by integrating visual perception and working memory, the brain's short-term storage of relevant visual information. The project will provide a new computational framework for understanding perception and memory which challenges traditional theories.Technically, vision can be viewed as involving both encoding and decoding. Encoding refers to how visual stimuli evoke sensory responses in the brain whereas decoding concerns how these responses eventually lead to the subjective perception of the stimuli. A common assumption of many existing models is that decoding follows the same low-to-high-level hierarchy as encoding, but this was never rigorously tested. Additionally, under natural viewing conditions, the small fovea and frequent saccades introduce delays between sensory encoding of different parts of a scene and perceptual integration of the whole scene, suggesting that working memory must be involved in perceptual decoding; yet previous decoding models do not consider working memory. This project aims to address these issues using psychophysical and computational methods, with the specific goal of elucidating the nature of decoding hierarchy in light of working-memory properties. Specifically, compared with lower-level stimulus features, higher-level features are more invariant and categorical, thus requiring less information to specify and permitting more stable maintenance in noisy working memory. The brain should therefore prioritize decoding of reliable higher-level features and then use them to constrain and improve the decoding of unstable lower-level features in memory (when necessary). The project will test some surprising predictions of this retrospective Bayesian decoding theory and develop a neural network implementation of the theory. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在观看场景时，我们通常会对其高级类别（例如，家庭，办公室，街道或丛林）有快速准确的感知理解。我们很少关注场景的低级别属性，例如各个点的亮度值，除非我们被要求报告它们，即使这样，我们对它们也不是很准确。场景的高层次属性比低层次属性与我们的行为更相关，这一事实为感知的全局优先理论提供了信息。然而，对大脑的实验研究已经确定，场景中的低级特征在高级特征之前被检测到;这一结果不知何故导致了通常使用但很少测试的假设，即视觉感知遵循相同的从低级到高级的特征检测层次。该项目试图通过分离特征检测和感知，并通过整合视觉感知和工作记忆（大脑对相关视觉信息的短期存储）来解决这一明显的矛盾。该项目将为理解感知和记忆提供一个新的计算框架，挑战传统的理论。从技术上讲，视觉可以被视为涉及编码和解码。编码是指视觉刺激如何在大脑中引起感官反应，而解码则涉及这些反应最终如何导致对刺激的主观感知。许多现有模型的一个共同假设是解码遵循与编码相同的从低到高的层次结构，但这从未经过严格的测试。此外，在自然观看条件下，小中央凹和频繁的扫视引入延迟之间的感觉编码的不同部分的场景和整个场景的感知整合，这表明工作记忆必须参与感知解码，但以前的解码模型不考虑工作记忆。该项目旨在使用心理物理和计算方法来解决这些问题，具体目标是根据工作记忆特性阐明解码层次的性质。具体而言，与低级别的刺激特征相比，高级别的特征更具有不变性和分类性，因此需要更少的信息来指定，并允许在嘈杂的工作记忆中更稳定的维护。因此，大脑应该优先解码可靠的高级特征，然后用它们来限制和改善记忆中不稳定的低级特征的解码（必要时）。该项目将测试一些令人惊讶的预测，这种回顾贝叶斯解码理论和开发神经网络实现的理论。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

项目成果

Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes

用于连续变量工作记忆的循环神经网络模型：活动流形、连接模式和动态代码

• DOI: 10.48550/arxiv.2111.01275
• 发表时间: 2021
• 期刊: ArXivorg
• 作者: Cueva, Christopher J.;Ardalan, Adel;Tsodyks, Misha;Qian, Ning
• 通讯作者: Qian, Ning

Computational modeling of excitatory/inhibitory balance impairments in schizophrenia.

精神分裂症中兴奋性/抑制平衡障碍的计算模型。

• DOI: 10.1016/j.schres.2020.03.027
• 发表时间: 2022-11
• 期刊: Schizophrenia research
• 影响因子: 4.5
• 作者: Qian N;Lipkin RM;Kaszowska A;Silipo G;Dias EC;Butler PD;Javitt DC
• 通讯作者: Javitt DC

Cross-fixation interactions of orientations suggest high-to-low-level decoding in visual working memory

方向的交叉注视相互作用表明视觉工作记忆中的高水平到低水平的解码

• DOI: 10.1016/j.visres.2021.107963
• 发表时间: 2022
• 期刊: Vision Research
• 影响因子: 1.8
• 作者: Luu, Long;Zhang, Mingsha;Tsodyks, Misha;Qian, Ning
• 通讯作者: Qian, Ning

Neuronal Firing Rate as Code Length: A Hypothesis

神经元放电率作为代码长度：一个假设

• 发表时间: 2019
• 期刊: Computational brain & behavior
• 作者: Qian, Ning;Zhang, Jun
• 通讯作者: Zhang, Jun