Combining Model- and Irradiance-based Constraints for Improved Face Shape Recovery and Recognition From Single Images

从单幅图像中估计三维人脸形状的问题近年来引起了人们的广泛关注。其原因之一是3D形状信息提供了面部的姿势和照明不变描述，其可以直接用于识别，或者产生照明和姿势归一化图像以输入到2D识别系统。最近的工作，已执行识别使用通过非标准的传感方式获得的三维人脸形状信息已经证明了潜在的好处3D超过2D强度图像。其中最重要的是提高了对姿势、照明和表情变化的鲁棒性。然而，在这些系统中，使用面部而不是其他生物特征的好处被丢失了，因为它们需要主体的参与和知识。另一方面，仅需要单个强度图像作为输入的系统可以部署在现有设备捕获图像数据的任何场景中，例如CCTV摄像机。至关重要的是，这样的系统可以用来识别只有一个训练图像存在的受试者。因此，很明显，人脸识别技术的一些最有吸引力的应用是通过从单个图像中恢复人脸形状的鲁棒方法来实现的。这一领域的最新进展遵循两种不同的研究途径。第一个使用的面部外观的变形模型，而第二个使用经典的形状从阴影技术。使用可变形模型的工作主体提供了从真实的图像的全局精确和鲁棒的形状恢复，但是将模型拟合到图像在计算上是昂贵的，并且恢复的形状缺乏局部细节，并且仅弱地满足图像辐照度约束（并且因此是照相现实主义）。最近的结果表明，从阴影形状可以从人脸图像中恢复精细的局部表面细节，但估计的全局3D形状的质量较差，使用变形模型恢复。这项建议的主要目的是研究介于这些目前不同的研究领域之间的新方法。在这样做时，希望能够将两种方法的优点联合收割机结合起来，同时解决上述缺点。我们还打算探索与投射阴影形成的统计建模相关的研究途径，并重新审视可变形模型构建的过程。最后，我们将这些增强的技术应用于具有挑战性的人脸识别任务。

The problem of estimating 3-dimensional face shape from single images has attracted considerable attention in recent years. One of the reasons for this is that 3D shape information provides a pose and illumination invariant description of a face, which can either be used for recognition directly, or to produce illumination and pose normalised images for input to a 2D recognition system. Recent work which has performed recognition using 3D face shape information acquired through non-standard sensing modalities has demonstrated the potential benefits of 3D over 2D intensity images. The most important of these are improved robustness to changes in pose, illumination and expression. However, the benefits of using the face over other biometrics are lost in these systems since they require the participation and knowledge of a subject. On the other hand, systems which require only a single intensity image as input may be deployed in any scenario where existing equipment captures image data, for example CCTV cameras. Crucially, such a system could be used to recognise subjects for whom only a single training image exists. It is therefore clear that some of the most alluring applications of face recognition technology are made possible by robust methods for face shape recovery from single images. Recent advances in this area have followed two distinct avenues of research. The first uses a morphable model of facial appearance while the second uses classical shape-from-shading techniques. The body of work which uses morphable models offers globally accurate and robust shape recovery from real images, but fitting the model to an image is computationally expensive and the recovered shape lacks local detail and only weakly satisfies image irradiance constraints (and hence photorealism). Recent results in shape-from-shading have shown that fine local surface detail can be recovered from face images but that the estimated global 3D shape is of inferior quality to that recovered using a morphable model. The primary aim of this proposal is to research new methods which will lie between these currently divergent strands of research. In doing so, the hope is to combine the strengths of both approaches while tackling the weaknesses described above. We also intend to explore avenues of research relating to statistical modelling of cast shadow formation and revisiting the process of morphable model construction. Finally we will apply these enhanced techniques to challenging face recognition tasks.