Skin Reflectance and Face Shape Estimation Using Photometric Stereo

本提案旨在通过一种新颖、非侵入性和高效的皮肤反射捕获技术来推进最先进的3D人脸识别技术。开发的技术将反过来实现快速面部几何分析和提高识别率。人脸识别是目前工业和学术界快速发展的研究领域。事实上，2D人脸识别目前正处于一个可以应用于一些工业领域的阶段。然而，这些方法只使用一张人脸的2D图像来进行识别，当存在姿势、照明、化妆或表情等变化时，人脸会变得无法识别，这一事实受到了过度的限制。然而，面部的3D形状并不会随着这些变化而改变，而且随着表情的变化也很少。因此，越来越多的人脸识别研究集中在如何利用人脸的三维形状进行识别。在这里，我们提出了一种用于三维形状估计的光度立体（PS）方法。与其他3D面部形状捕获设备相比，所提出的方法的主要优点是：(1)构建硬件更便宜，(2)快速采集和处理，(3)基本上不受环境光照的影响，(4)考虑个人特定反射率，(5)比标准PS更准确，(6)使用反射率特性帮助识别的可能性，以及(7)所需的校准最少。科学文献中提出了大量使用三维面部几何的方法，并取得了非常有希望的结果。然而，如何在识别之前捕获受试者的3D面部形状是一个开放的问题。现有的方法使用的技术对大多数应用程序来说过于昂贵和太慢。这项建议的动机是需要解决这个问题。提出的工作的主要贡献将在两个领域：光度立体（PS）和反射分析。光度立体是一种通过在三个或多个照明方向下对物体进行成像来估计物体三维几何形状的方法。在这个项目中，我们将使用五个光源，目的是同时获取形状和反射率信息。我们将使用UWE开发的高速光相机同步设备来完成这项任务。这将允许在恢复表面的方向和测量的像素强度之间推导映射，这将形成皮肤反射率特性的定量测量。然后使用迭代方法来更新表面估计和反射率属性，直到收敛。因此，我们将得到一个反射率测量的查找表集和一个最佳形状估计，这将允许改进人脸识别。这是一种新颖的PS方法，应该允许我们减少当前大多数方法对表面方向施加的一些强假设。这里的主要挑战将是形成基于图像的皮肤反射率测量与整个面部皮肤方向之间的关系，以获得最佳的3D形状估计。该项目的最后阶段将涉及应用以前在MVL和其他机构开发的面部识别方法进行比较分析。与使用标准PS和其他技术的3D方法相比，这将证明识别率的改进。

This proposal aims to advance the state-of-the-art in 3D face recognition by means of a novel, non-intrusive and highly efficient skin reflectance capture technology. The techniques developed will, in-turn, enable rapid facial geometry analysis and enhanced recognition rates.Face recognition is currently a rapidly growing area of research within industry and academia. Indeed, 2D face recognition is now at a stage where a few industrial applications are possible. However, these methods, which just use a single 2D image of a face to perform the recognition, are excessively limited by the fact that the face becomes unrecognisable when variations such as pose, illumination, make-up or expression are present. However, the 3D shape of the face does not change at all with many of these variations, and changes only minimally with expression. Consequently, an increasing amount of face recognition research is focussing on ways to use the 3D shape of the face for identification.Here, we are proposing to use a Photometric Stereo (PS) method for 3D shape estimation. The main advantages of the proposed method compared to other 3D face shape capture devices will be (1) cheaper to construct hardware, (2) fast acquisition and processing, (3) largely unaffected by ambient illumination, (4) person-specific reflectance considered, (5) more accurate than standard PS, (6) possibility of using the reflectance properties to aid recognition, and (7) minimal calibration required.A large number of methods for using the 3D facial geometry have been proposed in the scientific literature and very promising results have been attained. However, the question of how to capture a subject's 3D face shape prior to recognition is an open one. Existing approaches use technology that is too expensive and too slow for most applications. This proposal is motivated by the need to address this question.The main contributions of the proposed work will be in two areas: photometric stereo (PS) and reflectance analysis. Photometric stereo is a method of estimating the 3D geometry of an object by imaging it under three or more illumination directions. For this project, we will be using five light sources, and aim to simultaneously acquire both shape and reflectance information. We will be using a high speed light-camera synchronisation device developed here at UWE for this task. This will allow deducing a mapping between the orientations of the recovered surface and the measured pixel intensities which will form a quantitative measure of the skin reflectance properties. An iterative method will then be used to update the surface estimate and the reflectance properties until convergence. Thus, we will arrive at a lookup-table set of reflectance measurements and an optimal shape estimate, which will allow for improved face recognition. This is a novel approach to PS and should allow us to diminish some of the strong assumptions on surface orientation that most current methods impose. The main challenge here will be in forming the relationships between the image-based skin reflectance measurements and the skin orientation for the whole face in order to acquire the optimal 3D shape estimate.The final stage of the project will involve applying face recognition methods developed previously both at the MVL and at other institutions for a comparative analysis. This will demonstrate improvements in recognition rates compared to 3D methods using standard PS and other techniques.