Seeing it all: Convolutional network layers map the function of the human visual system
Seeing it all: Convolutional network layers map the function of the human visual system
复制标题
DOI：
10.1016/j.neuroimage.2016.10.001
复制
发表时间：
2017-05-15
期刊：
影响因子：
5.7
通讯作者：
Thirion, Bertrand
中科院分区：
文献类型：
--
作者：
Eickenberg, Michael;Gramfort, Alexandre;Thirion, Bertrand
用于计算机视觉的卷积网络代表在哺乳动物视觉系统中执行计算的候选模型。我们使用它们作为在观看自然图像期间人类大脑活动的详细模型，通过基于它们的不同层和BOLD fMRI激活来构建预测模型。跨层分析预测性能会产生每个视觉大脑区域的特征指纹：早期视觉区域由较低级别的卷积网络层更好地描述，而后期视觉区域由较高级别的网络层更好地描述，表现出腹侧和背侧流的进展。我们的预测模型超越了大脑对自然图像的反应。我们在两个实验中说明了这一点，即retinotopy和脸的位置对立，通过合成大脑活动，并进行经典的脑mapping上it.The合成恢复相应的功能磁共振成像研究中观察到的激活，表明这种深度编码模型捕捉的大脑功能的表征，是跨实验范式的普遍性。
Convolutional networks used for computer vision represent candidate models for the computations performed in mammalian visual systems. We use them as a detailed model of human brain activity during the viewing of natural images by constructing predictive models based on their different layers and BOLD fMRI activations. Analyzing the predictive performance across layers yields characteristic fingerprints for each visual brain region: early visual areas are better described by lower level convolutional net layers and later visual areas by higher level net layers, exhibiting a progression across ventral and dorsal streams. Our predictive model generalizes beyond brain responses to natural images. We illustrate this on two experiments, namely retinotopy and face place oppositions, by synthesizing brain activity and performing classical brain mapping upon it. The synthesis recovers the activations observed in the corresponding fMRI studies, showing that this deep encoding model captures representations of brain function that are universal across experimental paradigms.