Face Recognition using Photometric Stereo

我们建议构建一个3D人脸识别系统。为了绕过传统的立体(需要首先解决匹配问题)、结构光(不提供颜色信息)和具有光谱不同光源的光度立体(这依赖于均匀着色成像对象的假设)的问题，我们建议使用光度立体来进行人脸重建。光度立体(PS)可以在每个像素的基础上重现结构细节和颜色，这是其他3D系统无法做到的。建议的方案将适用于用于认证目的的受控环境，但也适用于一般环境，例如公共活动的入场。我们将使用两种方法：从数据中重建表面和从PS集合中直接提取面部特征。在第一种方法中，一旦恢复了表面正常和反照率，就可以在任意新的姿势和照明条件下合成呈现人脸的图像，以允许新的观看条件。我们还打算在识别过程中使用一种新的多尺度面部特征匹配方法，其中的面部特征包括从总体面部和头部形状到精细皮肤的皮肤地形、反射率和纹理。后者可以被认为是一种详细的表面凹凸贴图形式，形成独特的皮肤印记或签名，并代表一种新的方法。因此，3D形状和2D强度数据都将用于识别或验证任务。我们建议使用可伸缩的方法进行匹配，这样我们就可以处理大型数据库。3D匹配将使用新提出的基于FFT互相关的入侵者算法进行，并将使用特征和分类器组合来进行更详细的匹配。我们方法的新颖性在于使用PS来提取3D信息，使用详细的面部特征，如痣、划痕和皮肤纹理，以及系统的设计，以便它可以在人移动时操作，以最小的干扰和最高的效率。我们有两个行业合作伙伴，他们将在系统设计、数据收集和开发以及内政部的支持方面做出贡献。我们将在三种可能的场景下对我们的系统进行评估：在人群中搜索的人脸(实时人脸识别)，必须识别的人(离线人脸识别)，以及必须对照声称的身份进行检查的人(人脸身份验证)。我们将在12个月内在我们的一个工业合作伙伴的办公室安装第一个原型系统，以便收集数据。我们设想的是一个门状的结构，当一个人走过时，灯会接连闪烁，而相机正在捕捉图像。我们建议从重建的效率和准确性方面研究最佳的灯光数量，以及使用不可见光来避免对闪光敏感的人的问题。我们还将调查必须捕捉的细节与建筑几何形状之间的关系。

We propose to construct a system for 3D face recognition. We propose to use photometric stereo for face reconstruction in order to by pass the problems of conventional stereo (that needs to solve the matching problem first), structured light (that does not supply colour information) and photometric stereo with spectrally distinct light sources (that relies on the assumption of uniformly coloured imaged objects). Photometric stereo (PS) can reproduce structural details and colour on a per pixel basis in a way that no other 3D system can. The proposed scheme will be appropriate for use in a controlled environment for authentication purposes, but also in a general environment e.g. the entrance of a public event. We shall use two routes: surface reconstruction from the data and direct extraction of facial characteristics from the PS set. In the first approach, once surface normal and albedo is recovered, images of the face may be synthetically rendered under arbitrary new pose and illumination conditions to allow novel viewing conditions. We also aim to use a new multi-scale facial feature matching approach in the recognition process, where facial features range from overall face and head shape to fine skin dermal topography, reflectance and texture. The latter may be thought of as a form of detailed surface bump map forming a unique skin-print or signature and represents a new approach. Hence both the 3D shape and 2D intensity data will be used in recognition or authentication tasks. We propose to use scalable methods for matching, so we can cope with large databases. 3D matching will be done with the newly proposed invaders algorithm which is FFT cross-correlation based, and more detailed matching will be done by using features and classifier combination. The novelty of our approach lies in the use of PS to extract 3D information, the use of detailed facial characteristics like moles, scratches, and skin texture, and in the design of the system so that it can operate while the person is moving, with minimum intrusion and maximum efficiency. We have two industrial collaborators who will contribute in system design, data gathering and exploitation and support from the Home Office. We shall evaluate our system following three possible scenaria: a face searched in the crowd (real time face recognition), a person has to be identified (off-line face recognition) and a person has to be checked against a claimed identity (face authentication). We shall install the first prototype system in the offices of one of our industrial partners in month 12, so that data can be collected. We envisage a door like structure with lights flashing in succession as a person walks through, while a camera is capturing images. We propose to investigate the optimal number of lights in terms of efficiency and accuracy of the reconstruction, and the option of using non-visible light to avoid problems with people sensitive to flashes. We shall also investigate the relationship between detail that has to be captured and the geometry of the construction.