Visual Perception as Retrospective Bayesian Decoding from High- to Low-level Features in Working Memory

在观看场景时，我们通常对其高级类别有快速而准确的感知理解，例如，家庭、办公室、街道或丛林。我们很少注意场景的低级别属性，如各个点的亮度值，除非我们被要求报告这些属性，即使这样，我们也不是非常准确。场景的高级属性比低级属性与我们的行为更相关这一事实已经形成了感知的全球优先理论。然而，对大脑的实验研究已经确定，场景中较低级别的特征在较高级别的特征之前被检测到；这一结果以某种方式导致了普遍使用但很少被检验的假设，即视觉感知遵循相同的从低到高级别的特征检测层次。这个项目试图通过分离特征检测和感知，并通过整合视觉感知和工作记忆，即大脑对相关视觉信息的短期存储，来解决这一明显的矛盾。该项目将为理解感知和记忆提供一个新的计算框架，挑战传统理论。从技术上讲，视觉可以被视为同时涉及编码和解码。编码指的是视觉刺激如何在大脑中唤起感觉反应，而解码则涉及这些反应最终如何导致对刺激的主观感知。许多现有模型的一个共同假设是，解码遵循与编码相同的从低到高的层次结构，但这从未经过严格的测试。此外，在自然观看条件下，小中心凹和频繁的扫视在场景不同部分的感觉编码和整个场景的知觉整合之间引入了延迟，这表明工作记忆肯定参与了知觉解码；然而，以前的解码模型没有考虑工作记忆。该项目旨在使用心理物理和计算方法解决这些问题，具体目标是根据工作记忆属性阐明解码层次的性质。具体地说，与较低水平的刺激特征相比，较高水平的特征更具不变性和专属性，因此需要更少的信息来指定，并允许在嘈杂的工作记忆中更稳定地保持。因此，大脑应该优先解码可靠的较高级别的特征，然后使用它们来限制和改进记忆中不稳定的较低级别特征的解码(必要时)。该项目将测试这种回溯性贝叶斯解码理论的一些令人惊讶的预测，并开发该理论的神经网络实现。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

When looking at a scene, we typically have a quick and accurate perceptual understanding of its high-level category, for example, a home, office, street, or jungle. We rarely pay attention to the scene's low-level properties such as luminance values at various spots unless we are asked to report them, and even then, we are not very accurate about them. The fact that higher-level properties of a scene are more relevant to our behavior than low-level properties has informed global precedence theories of perception. However, experimental studies of the brain have established that lower-level features in a scene are detected before higher-level features; this result somehow led to the commonly used, but rarely tested, assumption that visual perception follows the same low-to-high-level hierarchy of feature detection. This project attempts to resolve this apparent contradiction by separating feature detection and perception, and by integrating visual perception and working memory, the brain's short-term storage of relevant visual information. The project will provide a new computational framework for understanding perception and memory which challenges traditional theories.Technically, vision can be viewed as involving both encoding and decoding. Encoding refers to how visual stimuli evoke sensory responses in the brain whereas decoding concerns how these responses eventually lead to the subjective perception of the stimuli. A common assumption of many existing models is that decoding follows the same low-to-high-level hierarchy as encoding, but this was never rigorously tested. Additionally, under natural viewing conditions, the small fovea and frequent saccades introduce delays between sensory encoding of different parts of a scene and perceptual integration of the whole scene, suggesting that working memory must be involved in perceptual decoding; yet previous decoding models do not consider working memory. This project aims to address these issues using psychophysical and computational methods, with the specific goal of elucidating the nature of decoding hierarchy in light of working-memory properties. Specifically, compared with lower-level stimulus features, higher-level features are more invariant and categorical, thus requiring less information to specify and permitting more stable maintenance in noisy working memory. The brain should therefore prioritize decoding of reliable higher-level features and then use them to constrain and improve the decoding of unstable lower-level features in memory (when necessary). The project will test some surprising predictions of this retrospective Bayesian decoding theory and develop a neural network implementation of the theory. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.