Representations of regular and irregular shapes by deep Convolutional Neural Networks, monkey inferotemporal neurons and human judgments.
Representations of regular and irregular shapes by deep Convolutional Neural Networks, monkey inferotemporal neurons and human judgments.
复制标题
DOI：
10.1371/journal.pcbi.1006557
复制
发表时间：
2018-10
影响因子：
4.3
通讯作者：
Vogels R
中科院分区：
文献类型：
--
作者：
Kalfas I;Vinken K;Vogels R
最近的研究表明，与任何其他现有的物体识别模型相比，深度卷积神经网络（CNN）模型在猕猴下颞叶（IT）皮层反应、人类腹侧流fMRI激活和人类物体识别方面表现出更高的表征相似性。这些研究采用物体的自然图像。长期以来的研究传统是利用抽象的形状来探测IT神经元的选择性。如果CNN模型提供了IT响应的现实模型，那么它们应该捕获这些形状的IT选择性。在这里，我们将CNN单元的激活与猕猴IT神经元的响应选择性和人类相似性判断的2D规则和不规则形状的刺激集进行比较。形状集由规则形状组成，这些规则形状在非偶然性质上有所不同，而不规则的、不对称的形状则具有弯曲或直线的边界。我们发现深度cnn （Alexnet， VGG-16和VGG-19）被训练来对自然图像进行分类，对这些形状的反应调节与IT神经元相似。未经训练的cnn具有与训练过的cnn相同的架构，但具有随机权重，在分类方面表现出比训练过的cnn更差的相似性。训练后的cnn和未训练的cnn之间的差异出现在深度卷积层，其中IT神经元的形状相关响应调制与训练后的cnn之间的相似性很高。与IT神经元不同，人类对相同形状的相似性判断与训练后的cnn的最后一层相关性最好。特别是，这些最深的层对直线和弯曲的不规则形状表现出更高的敏感性，类似于人类对形状的判断。综上所述，猕猴IT神经元的抽象形状相似性表征与cnn的深度卷积层之间具有高度可比性，而人类形状相似性判断与最深层之间的相关性更好。灵长类动物的下颞叶皮层（IT）被认为是视觉处理的最后阶段，它允许物体识别、识别和分类。电生理学研究表明，物体的形状是IT神经元反应模式的重要决定因素。在这里，我们研究深度卷积神经网络（cnn），被训练来对物体的自然图像进行分类，是否表现出与猕猴IT神经元相似的抽象形状的响应调制。对于三个最先进的cnn的训练和未训练版本，我们评估了一组2D形状在每个阶段的响应调制，并将其与猕猴IT神经元群体和人类形状相似性判断进行了比较。我们表明，当深度卷积CNN层被训练用于对自然图像进行分类时，2D抽象形状之间的相似度的it表示在这些层中发展。我们的研究结果表明，深度训练的CNN阶段与猕猴IT神经元的形状相似性表示之间存在高度对应关系，并且最后训练的CNN阶段与人类判断的形状相似性具有类似的对应关系。
Recent studies suggest that deep Convolutional Neural Network (CNN) models show higher representational similarity, compared to any other existing object recognition models, with macaque inferior temporal (IT) cortical responses, human ventral stream fMRI activations and human object recognition. These studies employed natural images of objects. A long research tradition employed abstract shapes to probe the selectivity of IT neurons. If CNN models provide a realistic model of IT responses, then they should capture the IT selectivity for such shapes. Here, we compare the activations of CNN units to a stimulus set of 2D regular and irregular shapes with the response selectivity of macaque IT neurons and with human similarity judgements. The shape set consisted of regular shapes that differed in nonaccidental properties, and irregular, asymmetrical shapes with curved or straight boundaries. We found that deep CNNs (Alexnet, VGG-16 and VGG-19) that were trained to classify natural images show response modulations to these shapes that were similar to those of IT neurons. Untrained CNNs with the same architecture than trained CNNs, but with random weights, demonstrated a poorer similarity than CNNs trained in classification. The difference between the trained and untrained CNNs emerged at the deep convolutional layers, where the similarity between the shape-related response modulations of IT neurons and the trained CNNs was high. Unlike IT neurons, human similarity judgements of the same shapes correlated best with the last layers of the trained CNNs. In particular, these deepest layers showed an enhanced sensitivity for straight versus curved irregular shapes, similar to that shown in human shape judgments. In conclusion, the representations of abstract shape similarity are highly comparable between macaque IT neurons and deep convolutional layers of CNNs that were trained to classify natural images, while human shape similarity judgments correlate better with the deepest layers. The primate inferior temporal (IT) cortex is considered to be the final stage of visual processing that allows for object recognition, identification and categorization of objects. Electrophysiology studies suggest that an object’s shape is a strong determinant of the neuronal response patterns in IT. Here we examine whether deep Convolutional Neural Networks (CNNs), that were trained to classify natural images of objects, show response modulations for abstract shapes similar to those of macaque IT neurons. For trained and untrained versions of three state-of-the-art CNNs, we assessed the response modulations for a set of 2D shapes at each of their stages and compared these to those of a population of macaque IT neurons and human shape similarity judgements. We show that an IT-like representation of similarity amongst 2D abstract shapes develops in the deep convolutional CNN layers when these are trained to classify natural images. Our results reveal a high correspondence between the representation of shape similarity of deep trained CNN stages and macaque IT neurons and an analogous correspondence of the last trained CNN stages with shape similarity as judged by humans.
登录
查看更多内容
影响因子：
5.7
作者：
Eickenberg, Michael;Gramfort, Alexandre;Thirion, Bertrand
通讯作者：
Thirion, Bertrand
DOI：
10.1111/j.2517-6161.1995.tb02031.x
发表时间：
1995-01-01
影响因子：
5.8
作者：
BENJAMINI, Y;HOCHBERG, Y
通讯作者：
HOCHBERG, Y
影响因子：
4.3
作者：
Khaligh-Razavi SM;Kriegeskorte N
通讯作者：
Kriegeskorte N
影响因子：
5.3
作者：
Guclu, Umut;van Gerven, Marcel A. J.
通讯作者：
van Gerven, Marcel A. J.
影响因子：
2.5
作者：
GOCHIN, PM;COLOMBO, M;GROSS, CG
通讯作者：
GROSS, CG