Visual imagery: an investigation of its neural and computational basis

当我们在脑海中想象出一个物体的形象时，我们是如何体验它的视觉外观的？视觉意象是许多人类活动的关键，如学习、回忆和规划。因此，这种认知能力在神经科学和心理学中引起了长期的兴趣和争论。然而，图像在神经层面上的精确机制仍有待阐明。当前提案的首要目标是开发视觉图像的神经计算帐户。具体来说，目前的提案旨在回答该领域的一些基本挑战。首先，它试图澄清视觉图像在人脑中的位置以及如何随着时间的推移而展开。二是描述图像体验的详细内容，并基于神经数据将其可视化。第三，它寻求开发一种计算模型，能够解释视觉图像及其与视觉感知的关系。值得注意的是，这项工作旨在为基于神经的图像重建开发新的方法，以恢复图像表征的内容（例如，将从记忆中恢复的朋友的脸或想象的字母串的外观可视化）。上述目标将主要在面部和视觉词识别领域内进行，因为它们与图像和感知研究的相关性和实用性。这项工作将依赖于心理物理测试、计算建模（即深度神经网络）和神经成像（特别是功能磁共振成像（fMRI）和脑电图（EEG））在健康成人中的结合。上面概述的工作有可能将视觉图像的研究置于一个新的计算基础上。理论上，这项工作可以加深我们对视觉认知的神经机制的理解。在方法上，它进一步发展了神经数据解码和基于神经的图像重建的分析工具。实际上，它为依赖于视觉图像的神经读出（例如，通过将相应的神经数据转换为视觉显示来分享个人的心理印象）的新型人际交流模式奠定了基础。

How do we experience the visual appearance of an object when we conjure up its image in the mind's eye? Visual imagery is key to many human activities, such as learning, reminiscing and planning. Accordingly, this cognitive ability has elicited long-standing interest and debate in neuroscience and psychology. Yet, the precise mechanisms by which imagery is carried out at the neural level remain to be elucidated. The overarching goal of the current proposal is to develop a neurocomputational account of visual imagery. Specifically, the current proposal aims to answer a number of fundamental challenges in the field. First, it seeks to clarify where and how visual imagery unfolds over time in the human brain. Second, it aims to describe the detailed content of imagery experiences and visualize it based on neural data. And third, it seeks to develop a computational model able to account for visual imagery and its relationship with visual perception. Of note, this work aims to develop novel methodology for neural-based image reconstruction in order to recover the content of imagery representations (e.g., visualizing the appearance of a friend's face as recovered from memory or that of an imagined letter string). The aims above will be pursued primarily within the areas of face and visual word recognition, given their relevance and utility to the study of imagery and perception. This work will rely on a combination of psychophysical testing, computational modeling (i.e., deep neural networks) and neuroimaging, specifically functional magnetic resonance imaging (fMRI) and electroencephalography (EEG), in healthy adults. The work outlined above has the potential to place the study of visual imagery on a novel computational footing. Theoretically, this work can deepen our understanding of neural mechanisms for visual cognition. Methodologically, it furthers the development of analytical tools for neural data decoding and for neural-based image reconstruction. Practically, it lays the groundwork for a novel mode of interpersonal communication relying on neural read-outs of visual imagery (e.g., sharing an individual's mental impression by the conversion of corresponding neural data into a visual display).