Face Recognition using Photometric Stereo

我们提出构建一个三维人脸识别系统。为了克服传统立体（需要首先解决匹配问题）、结构光（不提供颜色信息）和具有光谱不同光源的光度立体（依赖于假设成像物体的颜色均匀）的问题，我们建议使用光度立体进行人脸重建。光度立体（PS）可以在每个像素的基础上再现结构细节和颜色，这是其他3D系统无法做到的。建议的方案既适合在受控环境中用于身份验证，也适用于一般环境，例如公共活动的入口。我们将使用两种路径：从数据中进行表面重建和直接从PS集中提取面部特征。在第一种方法中，一旦恢复表面法线和反照率，人脸图像可以在任意新姿态和照明条件下合成渲染，以允许新的观看条件。我们还将在识别过程中使用一种新的多尺度面部特征匹配方法，其中面部特征范围从整体面部和头部形状到精细皮肤真皮地形，反射率和纹理。后者可以被认为是一种详细的表面凹凸图形式，形成独特的皮肤印记或签名，代表了一种新的方法。因此，三维形状和二维强度数据都将用于识别或认证任务。我们建议使用可扩展的方法进行匹配，这样我们就可以应对大型数据库。利用新提出的基于FFT互相关的入侵者算法进行三维匹配，并结合特征和分类器进行更细致的匹配。我们的方法的新颖之处在于使用PS提取3D信息，使用痣，划痕和皮肤纹理等详细的面部特征，以及系统的设计，使其可以在人移动时运行，以最小的侵入和最高的效率。我们有两名工业合作者，他们将参与系统设计、数据收集和开发，并得到内政部的支持。我们将在以下三种可能的情况下评估我们的系统：在人群中搜索人脸（实时人脸识别），必须识别一个人（离线人脸识别），必须根据所声称的身份检查一个人（人脸认证）。我们将于12月在我们的一个工业合作伙伴的办公室安装第一个原型系统，以便收集数据。我们设想一个像门一样的结构，当一个人走过时，灯光连续闪烁，而相机正在捕捉图像。我们建议在重建的效率和准确性方面研究最佳灯光数量，以及使用非可见光的选择，以避免对闪光灯敏感的人的问题。我们还将研究必须捕捉的细节与建筑几何之间的关系。

We propose to construct a system for 3D face recognition. We propose to use photometric stereo for face reconstruction in order to by pass the problems of conventional stereo (that needs to solve the matching problem first), structured light (that does not supply colour information) and photometric stereo with spectrally distinct light sources (that relies on the assumption of uniformly coloured imaged objects). Photometric stereo (PS) can reproduce structural details and colour on a per pixel basis in a way that no other 3D system can. The proposed scheme will be appropriate for use in a controlled environment for authentication purposes, but also in a general environment e.g. the entrance of a public event. We shall use two routes: surface reconstruction from the data and direct extraction of facial characteristics from the PS set. In the first approach, once surface normal and albedo is recovered, images of the face may be synthetically rendered under arbitrary new pose and illumination conditions to allow novel viewing conditions. We also aim to use a new multi-scale facial feature matching approach in the recognition process, where facial features range from overall face and head shape to fine skin dermal topography, reflectance and texture. The latter may be thought of as a form of detailed surface bump map forming a unique skin-print or signature and represents a new approach. Hence both the 3D shape and 2D intensity data will be used in recognition or authentication tasks. We propose to use scalable methods for matching, so we can cope with large databases. 3D matching will be done with the newly proposed invaders algorithm which is FFT cross-correlation based, and more detailed matching will be done by using features and classifier combination. The novelty of our approach lies in the use of PS to extract 3D information, the use of detailed facial characteristics like moles, scratches, and skin texture, and in the design of the system so that it can operate while the person is moving, with minimum intrusion and maximum efficiency. We have two industrial collaborators who will contribute in system design, data gathering and exploitation and support from the Home Office. We shall evaluate our system following three possible scenaria: a face searched in the crowd (real time face recognition), a person has to be identified (off-line face recognition) and a person has to be checked against a claimed identity (face authentication). We shall install the first prototype system in the offices of one of our industrial partners in month 12, so that data can be collected. We envisage a door like structure with lights flashing in succession as a person walks through, while a camera is capturing images. We propose to investigate the optimal number of lights in terms of efficiency and accuracy of the reconstruction, and the option of using non-visible light to avoid problems with people sensitive to flashes. We shall also investigate the relationship between detail that has to be captured and the geometry of the construction.