NeuroGen: Activation optimized image synthesis for discovery neuroscience.
NeuroGen: Activation optimized image synthesis for discovery neuroscience.
复制标题
DOI：
10.1016/j.neuroimage.2021.118812
复制
发表时间：
2022-02-15
期刊：
影响因子：
5.7
通讯作者：
Kuceyeski A
中科院分区：
文献类型：
--
作者：
Gu Z;Jamison KW;Khosla M;Allen EJ;Wu Y;St-Yves G;Naselaris T;Kay K;Sabuncu MR;Kuceyeski A
功能性磁共振成像（fMRI）是一种强大的技术，使我们能够表征视觉皮层对刺激的反应，但这种实验本质上是基于先验假设构建的，仅限于扫描仪中呈现给个体的图像集，在观察到的大脑反应中受到噪声的影响，并且可能在个体之间存在很大差异。在这项工作中，我们提出了一种新的计算策略，我们称之为NeuroGen，以克服这些限制，并为人类视觉神经科学的发现开发一个强大的工具。NeuroGen将fMRI训练的人类视觉神经编码模型与深度生成网络相结合，以合成预测实现宏观大脑激活目标模式的图像。我们证明了编码模型提供的噪声减少，再加上生成网络产生高保真图像的能力，导致视觉神经科学的强大发现架构。通过使用NeuroGen创建的少量合成图像，我们证明了我们可以检测和放大区域和个体人脑对视觉刺激的反应模式的差异。然后，我们验证了这些发现反映在几千个观察到的图像响应与功能磁共振成像测量。我们进一步证明，NeuroGen可以创建合成图像，预计可实现最佳匹配自然图像无法实现的区域响应模式。NeuroGen框架扩展了大脑编码模型的实用性，为探索和精确控制人类视觉系统开辟了新的途径。
Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network’s ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.
登录
查看更多内容
DOI：
10.1016/j.cogbrainres.2004.03.020
发表时间：
2004-08-01
期刊：
COGNITIVE BRAIN RESEARCH
影响因子：
--
作者：
Blonder, LX;Smith, CD;Andersen, AH
通讯作者：
Andersen, AH
影响因子：
3
作者：
Huth AG;Lee T;Nishimoto S;Bilenko NY;Vu AT;Gallant JL
通讯作者：
Gallant JL
影响因子：
64.8
作者：
Epstein, R;Kanwisher, N
通讯作者：
Kanwisher, N
影响因子：
25
作者：
Dworkin, Jordan D.;Linn, Kristin A.;Bassett, Danielle S.
通讯作者：
Bassett, Danielle S.
影响因子：
2.5
作者：
KOBATAKE, E;TANAKA, K
通讯作者：
TANAKA, K