The feature-weighted receptive field: an interpretable encoding model for complex feature spaces.
The feature-weighted receptive field: an interpretable encoding model for complex feature spaces.
复制标题
DOI：
10.1016/j.neuroimage.2017.06.035
复制
发表时间：
2018-10-15
期刊：
影响因子：
5.7
通讯作者：
Naselaris T
中科院分区：
文献类型：
--
作者：
St-Yves G;Naselaris T
我们引入了特征加权感受野（fwRF），一种旨在平衡表达性，可解释性和可扩展性的编码模型。的fwRF是围绕一个功能图的概念组织的视觉刺激到视觉功能，保持视觉空间的拓扑结构（但不一定是原生分辨率的刺激）的转换。fwRF模型的关键假设是每个体素中的活动编码跨多个特征图的空间局部区域中的变化。该区域对于所有特征图都是固定的;然而，每个特征图对体素活动的贡献是加权的。因此，该模型具有两组可分离的参数：表征在视觉特征上汇集的位置和程度的“在哪里”参数，以及表征对视觉特征的调谐的“是什么”参数。“where”参数类似于经典的感受野，而“what”参数类似于经典的调谐函数。通过将这些参数视为可分离的参数，fwRF模型的复杂性与底层特征图的分辨率无关。这使得从相对少量的数据中估计具有数千个高分辨率特征图的模型成为可能。一旦从数据中估计出fwRF模型，就可以直接读取空间池化和特征调整，而不需要（或很少）额外的后处理或计算机实验。我们描述了一种优化算法，用于从标准视觉神经成像实验中获得的数据中估计fwRF模型。然后，我们展示了该模型对两组不同特征的应用：Gabor小波和由深度卷积神经网络提供的特征。我们发现，当使用Gabor特征图时，fwRF模型恢复了与视觉皮层的已知组织原则一致的感受野和空间频率调谐函数。我们还表明，fwRF模型可以用来回归整个深度卷积网络对大脑活动的影响。在单个编码模型中使用整个网络的能力产生了最先进的预测精度。我们的研究结果表明，特征加权感受野模型有着广泛的用途，从自然场景的视网膜定位映射，到将整个深层神经网络的活动回归到测量的大脑活动。
We introduce the feature-weighted receptive field (fwRF), an encoding model designed to balance expressiveness, interpretability and scalability. The fwRF is organized around the notion of a feature map—a transformation of visual stimuli into visual features that preserves the topology of visual space (but not necessarily the native resolution of the stimulus). The key assumption of the fwRF model is that activity in each voxel encodes variation in a spatially localized region across multiple feature maps. This region is fixed for all feature maps; however, the contribution of each feature map to voxel activity is weighted. Thus, the model has two separable sets of parameters: “where” parameters that characterize the location and extent of pooling over visual features, and “what” parameters that characterize tuning to visual features. The “where” parameters are analogous to classical receptive fields, while “what” parameters are analogous to classical tuning functions. By treating these as separable parameters, the fwRF model complexity is independent of the resolution of the underlying feature maps. This makes it possible to estimate models with thousands of high-resolution feature maps from relatively small amounts of data. Once a fwRF model has been estimated from data, spatial pooling and feature tuning can be read-off directly with no (or very little) additional post-processing or in-silico experimentation. We describe an optimization algorithm for estimating fwRF models from data acquired during standard visual neuroimaging experiments. We then demonstrate the model’s application to two distinct sets of features: Gabor wavelets and features supplied by a deep convolutional neural network. We show that when Gabor feature maps are used, the fwRF model recovers receptive fields and spatial frequency tuning functions consistent with known organizational principles of the visual cortex. We also show that a fwRF model can be used to regress entire deep convolutional networks against brain activity. The ability to use whole networks in a single encoding model yields state-of-the-art prediction accuracy. Our results suggest a wide variety of uses for the feature-weighted receptive field model, from retinotopic mapping with natural scenes, to regressing the activities of whole deep neural networks onto measured brain activity.
登录
查看更多内容
影响因子：
19.9
作者：
Naselaris T;Kay KN
通讯作者：
Kay KN
影响因子：
9.2
作者：
Nishimoto, Shinji;Vu, An T.;Naselaris, Thomas;Benjamini, Yuval;Yu, Bin;Gallant, Jack L.
通讯作者：
Gallant, Jack L.
影响因子：
5.3
作者：
Guclu, Umut;van Gerven, Marcel A. J.
通讯作者：
van Gerven, Marcel A. J.
影响因子：
16.2
作者：
Stansbury DE;Naselaris T;Gallant JL
通讯作者：
Gallant JL
深度神经网络： 生物视觉和大脑信息处理建模的新框架
DOI：
10.1146/annurev-vision-082114-035447
发表时间：
2015-01-01
期刊：
ANNUAL REVIEW OF VISION SCIENCE, VOL 1
影响因子：
--
作者：
Kriegeskorte, Nikolaus
通讯作者：
Kriegeskorte, Nikolaus