Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition.
Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition.
复制标题
DOI：
10.1167/jov.21.8.15
复制
发表时间：
2021-08-02
影响因子：
1.8
通讯作者：
O'Toole AJ
中科院分区：
文献类型：
--
作者：
Parde CJ;Colón YI;Hill MQ;Castillo CD;Dhar P;O'Toole AJ
单一单位的反应和人口的代码不同的“读出”的信息，他们提供的高层次的视觉表示。分散的局部和全局读数可能难以与体内方法相协调。为了弥合这一差距，我们使用经过人脸识别训练的深度卷积神经网络（DCNN）研究了身份、性别和观点的单个代码和集成代码之间的关系。类似于灵长类动物的视觉系统，DCNN开发了对图像变化进行概括的表示，同时保留了主题（例如，性别）和图像（例如，观点）信息。在单元层面，我们测量了预测属性（身份，性别，观点）所需的单个单元的数量以及每个属性的单个单元的预测值。识别是非常准确的使用随机样本只有3%的网络的输出单元，所有的单位有很大的身份预测能力。跨单位的反应是最低限度的相关性，表明单一的单位代码非冗余的身份线索。性别和观点分类需要大规模汇集的单位，个别单位有较弱的预测能力。在整体水平上，人脸表征的主成分分析表明，身份，性别和观点分离到高维子空间，解释方差排序。基于单位的方向在代表性的空间进行了比较的方向与属性。身份，性别和观点有助于所有个人单位的反应，削弱了神经调谐类比。相反，单一单位的反应携带叠加，分布式代码的面孔身份，性别和观点。这破坏了对DCNN和高级视觉的单位响应曲线的神经表征解释的信心。
Single-unit responses and population codes differ in the “read-out” information they provide about high-level visual representations. Diverging local and global read-outs can be difficult to reconcile with in vivo methods. To bridge this gap, we studied the relationship between single-unit and ensemble codes for identity, gender, and viewpoint, using a deep convolutional neural network (DCNN) trained for face recognition. Analogous to the primate visual system, DCNNs develop representations that generalize over image variation, while retaining subject (e.g., gender) and image (e.g., viewpoint) information. At the unit level, we measured the number of single units needed to predict attributes (identity, gender, viewpoint) and the predictive value of individual units for each attribute. Identification was remarkably accurate using random samples of only 3% of the network's output units, and all units had substantial identity-predicting power. Cross-unit responses were minimally correlated, indicating that single units code non-redundant identity cues. Gender and viewpoint classification required large-scale pooling of units—individual units had weak predictive power. At the ensemble level, principal component analysis of face representations showed that identity, gender, and viewpoint separated into high-dimensional subspaces, ordered by explained variance. Unit-based directions in the representational space were compared with the directions associated with the attributes. Identity, gender, and viewpoint contributed to all individual unit responses, undercutting a neural tuning analogy. Instead, single-unit responses carry superimposed, distributed codes for face identity, gender, and viewpoint. This undermines confidence in the interpretation of neural representations from unit response profiles for both DCNNs and, by analogy, high-level vision.
登录
查看更多内容
DOI：
10.1126/science.1194908
发表时间：
2010-11-05
期刊：
Science (New York, N.Y.)
影响因子：
--
作者：
Freiwald WA;Tsao DY
通讯作者：
Tsao DY
影响因子：
5.3
作者：
Issa, Elias B.;DiCarlo, James J.
通讯作者：
DiCarlo, James J.
DOI：
10.1073/pnas.1721355115
发表时间：
2018-06-12
影响因子：
11.1
作者：
Phillips, P. Jonathon;Yates, Amy N.;O'Toole, Alice J.
通讯作者：
O'Toole, Alice J.
影响因子：
56.9
作者：
Bashivan, Pouya;Kar, Kohitij;DiCarlo, James J.
通讯作者：
DiCarlo, James J.
DOI：
10.1109/jrproc.1959.287207
发表时间：
1959-01-01
影响因子：
--
作者：
LETTVIN, JY;MATURANA, HR;PITTS, WH
通讯作者：
PITTS, WH