Visual imagery: an investigation of its neural and computational basis

当我们在脑海中想象一个物体的形象时，我们如何体验它的视觉外观？视觉意象是许多人类活动的关键，如学习，回忆和计划。因此，这种认知能力在神经科学和心理学中引起了长期的兴趣和争论。然而，意象在神经水平上的精确机制仍有待阐明。当前提案的首要目标是开发视觉图像的神经计算帐户。 具体而言，目前的建议旨在应对该领域的一些根本挑战。首先，它试图澄清视觉图像在哪里以及如何随着时间的推移在人脑中展开。其次，它旨在描述图像体验的详细内容，并基于神经数据将其可视化。第三，它试图开发一个计算模型，能够解释视觉意象及其与视觉感知的关系。值得注意的是，这项工作旨在开发基于神经的图像重建的新方法，以便恢复图像表示的内容（例如，可视化从记忆中恢复的朋友的脸的外观或想象的字母串的外观）。上述目标将主要在面部和视觉单词识别领域内实现，因为它们与图像和感知研究相关并具有实用性。这项工作将依赖于心理物理测试、计算建模（即，深度神经网络）和神经成像，特别是功能性磁共振成像（fMRI）和脑电图（EEG）。上面概述的工作有可能将视觉意象的研究置于一个新的计算基础上。从理论上讲，这项工作可以加深我们对视觉认知神经机制的理解。在方法上，它进一步发展了神经数据解码和基于神经的图像重建的分析工具。实际上，它为一种依赖于视觉图像的神经读出的新型人际交流模式奠定了基础（例如，通过将相应的神经数据转换成视觉显示来共享个体的心理印象）。

How do we experience the visual appearance of an object when we conjure up its image in the mind's eye? Visual imagery is key to many human activities, such as learning, reminiscing and planning. Accordingly, this cognitive ability has elicited long-standing interest and debate in neuroscience and psychology. Yet, the precise mechanisms by which imagery is carried out at the neural level remain to be elucidated. The overarching goal of the current proposal is to develop a neurocomputational account of visual imagery. Specifically, the current proposal aims to answer a number of fundamental challenges in the field. First, it seeks to clarify where and how visual imagery unfolds over time in the human brain. Second, it aims to describe the detailed content of imagery experiences and visualize it based on neural data. And third, it seeks to develop a computational model able to account for visual imagery and its relationship with visual perception. Of note, this work aims to develop novel methodology for neural-based image reconstruction in order to recover the content of imagery representations (e.g., visualizing the appearance of a friend's face as recovered from memory or that of an imagined letter string). The aims above will be pursued primarily within the areas of face and visual word recognition, given their relevance and utility to the study of imagery and perception. This work will rely on a combination of psychophysical testing, computational modeling (i.e., deep neural networks) and neuroimaging, specifically functional magnetic resonance imaging (fMRI) and electroencephalography (EEG), in healthy adults. The work outlined above has the potential to place the study of visual imagery on a novel computational footing. Theoretically, this work can deepen our understanding of neural mechanisms for visual cognition. Methodologically, it furthers the development of analytical tools for neural data decoding and for neural-based image reconstruction. Practically, it lays the groundwork for a novel mode of interpersonal communication relying on neural read-outs of visual imagery (e.g., sharing an individual's mental impression by the conversion of corresponding neural data into a visual display).