The contribution of object identity and configuration to scene representation in convolutional neural networks.
The contribution of object identity and configuration to scene representation in convolutional neural networks.
复制标题
DOI：
10.1371/journal.pone.0270667
复制
发表时间：
2022
期刊：
影响因子：
3.7
通讯作者：
Xu, Yaoda
中科院分区：
文献类型：
--
作者：
Tang, Kevin;Chin, Matthew;Chun, Marvin;Xu, Yaoda
场景感知涉及提取组成场景的对象的身份以及它们的配置(场景中对象的空间布局)。然而，在场景处理期间如何加权对象标识和配置信息以及该加权如何在场景处理过程中演变还没有完全理解。卷积神经网络(CNN)的最新发展已经证明了它们在场景处理任务中的能力，并确定了CNN和人脑中的处理之间的相关性。在这里，我们研究了四种CNN架构(Alexnet、Resnet18、Resnet50、Densenet161)，以及它们对场景处理过程中对象和配置信息变化的敏感性。尽管四种CNN架构不同，但在所有CNN中，我们观察到CNN对对象身份和配置变化的反应有一个共同的模式。每个CNN在加工的早期阶段对构型变化表现出更高的敏感性，在后期阶段对对象身份变化表现出更强的敏感性。无论图像背景中存在的空间结构、CNN对场景分类的准确性，甚至是用于训练CNN的任务，这种模式都会持续存在。重要的是，CNN对构型变化的敏感度与它们对任何类型的位置变化的敏感度不同，例如，由没有构型变化的对象的均匀平移引起的敏感度。这些结果提供了在场景处理期间对象身份和配置信息如何在CNN中加权的首批文献之一。
Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN’s response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs’ sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing.
登录
查看更多内容
影响因子：
64.8
作者：
Epstein, R;Kanwisher, N
通讯作者：
Kanwisher, N
影响因子：
5.7
作者：
King, Marcie L.;Groen, Iris I. A.;Baker, Chris I.
通讯作者：
Baker, Chris I.
影响因子：
5.3
作者：
Kourtzi, Z;Kanwisher, N
通讯作者：
Kanwisher, N
影响因子：
5.3
作者：
Park, Soojin;Brady, Timothy F.;Oliva, Aude
通讯作者：
Oliva, Aude
影响因子：
19.9
作者：
Dilks DD;Kamps FS;Persichetti AS
通讯作者：
Persichetti AS