Combining Model- and Irradiance-based Constraints for Improved Face Shape Recovery and Recognition From Single Images

从单幅图像中估计三维人脸形状的问题近年来引起了人们的广泛关注。其中一个原因是，3D形状信息提供了人脸的姿态和光照不变描述，可以直接用于识别，也可以生成光照和姿态归一化图像，输入到2D识别系统。最近的工作使用非标准传感模式获得的3D脸型信息进行识别，证明了3D比2D强度图像的潜在优势。其中最重要的是提高了对姿势、照明和表情变化的鲁棒性。然而，在这些系统中，使用人脸比其他生物识别技术的好处就消失了，因为它们需要受试者的参与和知识。另一方面，只需要单一强度图像作为输入的系统可以部署在现有设备捕获图像数据的任何场景中，例如闭路电视摄像机。至关重要的是，这样的系统可以用来识别只有一个训练图像存在的对象。因此，很明显，人脸识别技术的一些最诱人的应用是通过从单个图像中恢复人脸形状的鲁棒方法实现的。这一领域的最新进展遵循两种截然不同的研究途径。第一种方法使用可变形的面部外观模型，而第二种方法使用经典的阴影形状技术。使用可变形模型的工作主体从真实图像中提供全局精确和鲁棒的形状恢复，但是将模型拟合到图像上的计算成本很高，并且恢复的形状缺乏局部细节，并且只能弱地满足图像辐照度约束（因此是照片真实感）。最近的结果表明，形状从阴影可以从人脸图像中恢复良好的局部表面细节，但估计的全局3D形状的质量不如使用变形模型恢复的质量。这项提议的主要目的是研究介于这些目前分歧的研究分支之间的新方法。在这样做的过程中，希望结合这两种方法的优点，同时解决上述缺点。我们还打算探索与阴影形成的统计建模相关的研究途径，并重新审视可变形模型的构建过程。最后，我们将把这些增强的技术应用于具有挑战性的人脸识别任务。

The problem of estimating 3-dimensional face shape from single images has attracted considerable attention in recent years. One of the reasons for this is that 3D shape information provides a pose and illumination invariant description of a face, which can either be used for recognition directly, or to produce illumination and pose normalised images for input to a 2D recognition system. Recent work which has performed recognition using 3D face shape information acquired through non-standard sensing modalities has demonstrated the potential benefits of 3D over 2D intensity images. The most important of these are improved robustness to changes in pose, illumination and expression. However, the benefits of using the face over other biometrics are lost in these systems since they require the participation and knowledge of a subject. On the other hand, systems which require only a single intensity image as input may be deployed in any scenario where existing equipment captures image data, for example CCTV cameras. Crucially, such a system could be used to recognise subjects for whom only a single training image exists. It is therefore clear that some of the most alluring applications of face recognition technology are made possible by robust methods for face shape recovery from single images. Recent advances in this area have followed two distinct avenues of research. The first uses a morphable model of facial appearance while the second uses classical shape-from-shading techniques. The body of work which uses morphable models offers globally accurate and robust shape recovery from real images, but fitting the model to an image is computationally expensive and the recovered shape lacks local detail and only weakly satisfies image irradiance constraints (and hence photorealism). Recent results in shape-from-shading have shown that fine local surface detail can be recovered from face images but that the estimated global 3D shape is of inferior quality to that recovered using a morphable model. The primary aim of this proposal is to research new methods which will lie between these currently divergent strands of research. In doing so, the hope is to combine the strengths of both approaches while tackling the weaknesses described above. We also intend to explore avenues of research relating to statistical modelling of cast shadow formation and revisiting the process of morphable model construction. Finally we will apply these enhanced techniques to challenging face recognition tasks.