Visual imagery: an investigation of its neural and computational basis

当我们在头脑中想象一个物体的形象时，我们是如何体验它的视觉外观的？视觉意象是许多人类活动的关键，例如学习、回忆和计划。因此，这种认知能力在神经科学和心理学中引起了长期的兴趣和争论。然而，在神经水平上进行表象的确切机制仍有待阐明。当前提案的首要目标是开发一种视觉图像的神经计算帐户。具体地说，目前的提案旨在回答该领域的一些根本性挑战。首先，它试图澄清视觉图像在人脑中随着时间的推移在哪里以及如何展开。其次，它旨在描述表象体验的详细内容，并基于神经数据将其可视化。第三，它试图开发一种计算模型，能够解释视觉意象及其与视觉感知的关系。值得注意的是，这项工作旨在开发基于神经的图像重建的新方法，以便恢复图像表征的内容(例如，将从记忆中恢复的朋友的脸或想象的字母串的外观可视化)。上述目标将主要在人脸和视觉单词识别领域内实现，因为它们对图像和感知的研究具有相关性和实用性。这项工作将依赖于心理物理测试、计算建模(即深层神经网络)和神经成像的组合，特别是健康成年人的功能磁共振成像(FMRI)和脑电(EEG)。上面概述的工作有可能将视觉图像的研究置于一种新的计算基础上。从理论上讲，这项工作可以加深我们对视觉认知神经机制的理解。在方法上，它进一步发展了用于神经数据解码和基于神经的图像重建的分析工具。实际上，它为一种新的人际交流模式奠定了基础，这种模式依赖于视觉图像的神经读出(例如，通过将相应的神经数据转换为视觉显示来分享个人的心理印象)。

How do we experience the visual appearance of an object when we conjure up its image in the mind's eye? Visual imagery is key to many human activities, such as learning, reminiscing and planning. Accordingly, this cognitive ability has elicited long-standing interest and debate in neuroscience and psychology. Yet, the precise mechanisms by which imagery is carried out at the neural level remain to be elucidated. The overarching goal of the current proposal is to develop a neurocomputational account of visual imagery. Specifically, the current proposal aims to answer a number of fundamental challenges in the field. First, it seeks to clarify where and how visual imagery unfolds over time in the human brain. Second, it aims to describe the detailed content of imagery experiences and visualize it based on neural data. And third, it seeks to develop a computational model able to account for visual imagery and its relationship with visual perception. Of note, this work aims to develop novel methodology for neural-based image reconstruction in order to recover the content of imagery representations (e.g., visualizing the appearance of a friend's face as recovered from memory or that of an imagined letter string). The aims above will be pursued primarily within the areas of face and visual word recognition, given their relevance and utility to the study of imagery and perception. This work will rely on a combination of psychophysical testing, computational modeling (i.e., deep neural networks) and neuroimaging, specifically functional magnetic resonance imaging (fMRI) and electroencephalography (EEG), in healthy adults. The work outlined above has the potential to place the study of visual imagery on a novel computational footing. Theoretically, this work can deepen our understanding of neural mechanisms for visual cognition. Methodologically, it furthers the development of analytical tools for neural data decoding and for neural-based image reconstruction. Practically, it lays the groundwork for a novel mode of interpersonal communication relying on neural read-outs of visual imagery (e.g., sharing an individual's mental impression by the conversion of corresponding neural data into a visual display).