Deep Convolutional Neural Networks Outperform Feature-Based But Not Categorical Models in Explaining Object Similarity Judgments.
Deep Convolutional Neural Networks Outperform Feature-Based But Not Categorical Models in Explaining Object Similarity Judgments.
复制标题
DOI：
10.3389/fpsyg.2017.01726
复制
发表时间：
2017
影响因子：
3.8
通讯作者：
Mur M
中科院分区：
文献类型：
--
作者：
Jozwik KM;Kriegeskorte N;Storrs KR;Mur M
深度卷积神经网络（dnn）的最新进展使大脑表征的计算模型空前精确，并提供了一个令人兴奋的机会来模拟各种认知功能。最先进的深度神经网络在对象分类上实现了人类水平的表现，但它们在复杂的认知任务中如何捕捉人类行为尚不清楚。最近的报告表明，深度神经网络可以解释一个这样的任务中的显著差异，即判断物体的相似性。在这里，我们通过将这些发现复制到一组丰富的对象图像中，比较两个不同深度的dnn内的跨层性能，并检查dnn的性能与非计算性“概念”模型的性能比较来扩展这些发现。人类观察者对一组92张真实世界物体的图像进行相似性判断。在不同深度的两个dnn（8层AlexNet和16层VGG-16）的每一层中获得相同图像的表示。为了创建概念模型，其他人类观察者为同一图像集生成视觉特征标签（例如，“眼睛”）和类别标签（例如，“动物”）。特征标签分为部件、颜色、纹理和轮廓，类别标签分为下级、基本和上级类别。我们将从特征、类别和每个DNN的每一层导出的模型拟合到相似性判断中，使用表征相似性分析来评估模型的性能。在这两种深度神经网络中，最后一层的相似性解释了人类相似性判断中大部分可解释的差异。最后一层优于几乎所有基于特征的模型。后期和中层的性能优于一些但不是所有的基于特征的模型。重要的是，分类模型预测相似性判断明显优于任何深度神经网络层。我们的结果为dnn和大脑表征之间的共性提供了进一步的证据。来源于视觉特征而非物体部分的模型表现相对较差，这可能是因为dnn更全面地捕捉到与人类物体感知有关的颜色、纹理和轮廓。然而，分类模型的表现优于dnn，这表明可能需要进一步的工作来使dnn中的高级语义表示更接近人类提取的语义表示。现代dnn对相似性判断的解释非常好，因为它们没有接受过这方面的训练，并且在人类认知的许多方面都是有前途的模型。
Recent advances in Deep convolutional Neural Networks (DNNs) have enabled unprecedentedly accurate computational models of brain representations, and present an exciting opportunity to model diverse cognitive functions. State-of-the-art DNNs achieve human-level performance on object categorisation, but it is unclear how well they capture human behavior on complex cognitive tasks. Recent reports suggest that DNNs can explain significant variance in one such task, judging object similarity. Here, we extend these findings by replicating them for a rich set of object images, comparing performance across layers within two DNNs of different depths, and examining how the DNNs’ performance compares to that of non-computational “conceptual” models. Human observers performed similarity judgments for a set of 92 images of real-world objects. Representations of the same images were obtained in each of the layers of two DNNs of different depths (8-layer AlexNet and 16-layer VGG-16). To create conceptual models, other human observers generated visual-feature labels (e.g., “eye”) and category labels (e.g., “animal”) for the same image set. Feature labels were divided into parts, colors, textures and contours, while category labels were divided into subordinate, basic, and superordinate categories. We fitted models derived from the features, categories, and from each layer of each DNN to the similarity judgments, using representational similarity analysis to evaluate model performance. In both DNNs, similarity within the last layer explains most of the explainable variance in human similarity judgments. The last layer outperforms almost all feature-based models. Late and mid-level layers outperform some but not all feature-based models. Importantly, categorical models predict similarity judgments significantly better than any DNN layer. Our results provide further evidence for commonalities between DNNs and brain representations. Models derived from visual features other than object parts perform relatively poorly, perhaps because DNNs more comprehensively capture the colors, textures and contours which matter to human object perception. However, categorical models outperform DNNs, suggesting that further work may be needed to bring high-level semantic representations in DNNs closer to those extracted by humans. Modern DNNs explain similarity judgments remarkably well considering they were not trained on this task, and are promising models for many aspects of human cognition.
登录
查看更多内容
影响因子：
9.8
作者：
Haushofer, Johannes;Livingstone, Margaret S.;Kanwisher, Nancy
通讯作者：
Kanwisher, Nancy
影响因子：
4.3
作者：
Nili H;Wingfield C;Walther A;Su L;Marslen-Wilson W;Kriegeskorte N
通讯作者：
Kriegeskorte N
影响因子：
19.5
作者：
Russakovsky, Olga;Deng, Jia;Fei-Fei, Li
通讯作者：
Fei-Fei, Li
影响因子：
4.3
作者：
Khaligh-Razavi SM;Kriegeskorte N
通讯作者：
Kriegeskorte N
影响因子：
3.8
作者：
Kriegeskorte N;Mur M
通讯作者：
Mur M