Human Face Representation in Deep Convolutional Neural Networks

人类视觉系统可以在视点、照明、表情和外观的广泛变化中识别熟悉的面孔。这一非凡的计算壮举是由大规模的神经元网络完成的。我们将测试在深度学习卷积神经网络（DCNN）顶层出现的表示的人脸空间理论，作为人脸的人类视觉表示模型。近年来，由于DCNN和网络上标记的训练数据（人脸和身份）的容易获得，基于计算机的人脸识别得到了改善。受灵长类动物视觉系统的启发，DCNN是前馈人工神经网络，可以将人脸图像映射为支持识别广泛变化图像的表示。虽然模拟神经元执行的计算很简单，但将图像转换为表示需要大量的计算。该处理的最终结果是高度紧凑的面部表示，其以不变的、身份识别特定的面部代码保留图像细节。这种代码与视觉科学中考虑的任何面部表示都有根本的不同。我们测试的这个理论将以前的人脸空间模型的关键组成部分（相似性，学习历史）与新特征（成像条件，个人人脸历史）结合在一个统一的空间中，该空间代表了不同图像的身份和面部外观。我们将测试该模型是否可以解释人类对熟悉面孔的识别，这对图像变化（姿势，照明，表情）具有高度鲁棒性。该模型也将被应用于理解人类（和机器）与其他种族的面孔长期存在的困难。我们的目标是弥合我们对DCNN如何工作的知识的关键差距，将心理，神经和计算观点联系起来。一个全新的面部表征理论将改变我们在这三个领域中对面部表征的问题。一个新的焦点是理解我们（或神经网络）如何在广泛变化的图像中“感知”一个熟悉的身份，这将导致人们寻找一种表征，这种表征将面部的属性与他们所经历的真实的虚拟世界图像条件优雅地融合在一起。该项目提供了一个独特的机会来研究，操纵和学习这些表征，并将研究结果应用于从神经和感知角度研究高级视觉的更广泛问题。

The human visual system can recognize a familiar face across wide variations of viewpoint, illumination, expression, and appearance. This remarkable computational feat is accomplished by large-­scale networks of neurons. We will test a face space theory of the representations that emerge at the top layer of deep learning convolutional neural networks (DCNNs) as a model of human visual representations of faces. Computer-based face recognition has improved in recent years due to DCNNs and the easy availability of labeled training data (faces and identities) from the web. Inspired by the primate visual system, DCNNs are feed­forward artificial neural networks that can map images of faces into representations that support recognition over widely variable images. Although the calculations executed by the simulated neurons are simple, enormous numbers of computations are used to convert an image into a representation. The end result of this processing is a highly compact representation of a face that retains image detail in an invariant, identity­-specific face code. This code is fundamentally different than any representation of faces considered in vision science. This theory we test combines key components of previous face space models (similarity, learning history) with new features (imaging conditions, personal face history) in a unitary space that represents both identity and facial appearance across variable images. We will test whether this model can account for human recognition of familiar faces, which is highly robust to image variability (pose, illumination, expression). The model will also be applied to understanding long standing difficulties humans (and machines) have with faces of other races. We aim to bridge critical gaps in our knowledge of how DCNNs work, linking psychological, neural, and computational perspectives. A fundamentally new theory of face representation will alter the questions we ask about face representations in all three fields. A new focus on understanding how we (or neural networks) “perceive” a single familiar identity in widely variable images will give rise to a search for representations that gracefully merge the properties of faces with the real-­world image conditions in which they are experienced. This project presents a unique opportunity to study, manipulate, and learn from these representations, and to apply the findings to broader questions about high-­level vision from neural and perceptual perspectives.