Estimating the Directions to Light Sources Using Images of Eye for Reconstructing 3D Human Face
Estimating the Directions to Light Sources Using Images of Eye for Reconstructing 3D Human Face
复制标题
使用眼睛图像估计光源方向以重建 3D 人脸
DOI：
10.2352/cic.2003.11.1.art00014
复制
发表时间：
2003
影响因子：
0.7
通讯作者：
Y. Miyake
中科院分区：
文献类型：
--
作者：
N. Tsumura;Minh Dang;Y. Miyake
本文提出了一种基于人眼图像估计光源方向的方法，其中光源被成像为反射。利用估计的光源方向，基于光度立体技术重建人脸的三维形状。利用重建的三维形状，我们可以在不同的光源和不同的视角下再现人脸。在不知道被摄对象的站立位置的情况下，我们可以从三个光源的眼睛图像中估计出人脸上的光源方向。在此估计过程中，假设人眼为球体，并在成像系统中使用反射的几何约束。人眼的位置和大小也被估计为这个过程的结果。由于拍摄对象的站立位置在捕捉图像时不受限制，因此我们的成像系统使用起来非常实用。实验证明了所提方法的有效性。在不同的光源和不同的视点下，人脸的再现需要重构人脸的形状。为了在不使用昂贵的3D扫描仪的情况下获得人脸形状，采用了光度立体[3]，这是一种利用在不同光照条件下获得的一些强度图像进行形状估计的方法。然而，在光度立体法中，如果没有光源方向和光源强度的先验知识，就无法估计表面法线和表面反射率。因此，为了准确地得到光源照射在人脸上的方向，有必要确定光源与被摄对象位置的关系。本文提出了一种利用人眼图像来估计光源在人脸上的反射方向的方法。由于人眼的形状几乎是球形的，可以反射光线，我们可以假设人眼是一个镜像球来捕捉环境光源。根据镜像球技术[1,2]，利用镜像球图像中光源被镜像球反射的高光峰位置，可以正确计算出光源的方向。但是，这种方法需要精确的镜像球的位置和半径。由于眼睛的半径是人的，我们不容易测量它。此外，受试者和他的人眼的位置都没有确定。因此，如果不知道人眼的位置和半径，很难估计光源的方向。本文还从人眼图像中估计人眼的位置和半径。在这个估计中，我们假设三个光源在相机协调中的位置是已知的。然而，我们不知道拍摄对象站在或坐在成像系统前面的什么位置。利用估计的光源方向，利用光度立体法可以估计人脸的表面法线和表面反射率。我们可以通过在视图坐标中对估计的表面法线进行积分来重建三维人脸。实验证明了该方法的有效性。镜像眼的几何模型如图1所示。该成像过程基于镜像球技术的过程[1,2]。由于人脸的位置不受限制，人眼会被放置在自然环境中的任何位置。摄像机假定为针孔摄像机。因此，我们可以假设成像过程就像在相机前面放置了一个屏幕一样。相机与屏幕的距离为焦距。图1显示了3D空间中人脸通过摄像头在屏幕上的投影。图中N为高光峰值处的表面法线，L为人眼上的光源方向矢量，V为高光峰值处相机的方向矢量。D为相机与人眼之间的距离，（）0 0，y x为屏幕中心。摄像机在图像上第i个高光峰（）i i y x处的方向矢量可以用焦距α和（）0 0 0，y x表示如下图1。所提出的成像系统的几何形状[]i i y y x x α−−−=,,0 0 V。(1)由于图像的中心坐标已知，可以从图像中得到高光峰（）i i y x的坐标。通过标定得到了相机的焦距α。摄像机的方向很容易计算，如图1所示。曲面法向量N是用球面性质确定的。设r为眼睛的半径，（）c c y x为眼睛在图像上的中心坐标。第i个高光峰处的曲面法向量（）i i y x，可由
This paper proposes a technique to estimate the direction of light sources based on the image of eye where the light source is imaged as reflection. The estimated directions of light sources are used to reconstruct 3D shape of face based on the photometric stereo technique. By using the reconstructed 3D shape, we can reproduce faces under various illuminants and from various viewing points. Without knowing the standing position of the subject, we can estimate the directions of light sources on human face from there images of eye for three light sources. In the process of this estimation, it is assumed that human eye is the sphere, and the geometrical constrains for reflection are used in the imaging system. The position and the size of human eye are also estimated as a result of the process. Since the standing position of the subject taken is not restricted in capturing the images, our imaging system is very practical to be used. The effectiveness of the proposed techniques are demonstrated by experiments. Introduction Reconstructing shape of human face has been required to reproduce faces under various illuminants and from various viewing points. In order to obtain the shape of human face, without using the expensive 3D scanner, the photometric stereo[3], which is a method for shape estimation that uses some intensity images obtained under different lighting conditions, has been used. However, with photometric stereo method, it is impossible to estimate the surface normal and the surface reflectance without a priori knowledge of the light source direction and the light source intensity. Therefore, in order to get the direction of light sources on human face exactly, it is necessary to determine the relation between the positions of light sources and subject. In this paper, we propose a method for estimating the direction of light source on human face by using the image of eye where the light source is imaged as reflection. Since the shape of human eye is almost sphere, and can reflect light, we can assume that human eye is used as mirrored ball to capture the environmental illuminants. According to mirrored ball technique[1,2], using the position of highlight peaks in the image of mirrored ball, where the light sources are reflected by on mirrored ball, the direction of light sources can be calculated correctly. However, this method requires the accurate position and radius of the mirrored ball. Since the radius of eye belongs with people, we cannot measure it easily. Moreover, the position of both subject and his human eye are not decided. Therefore it is difficult to estimate the direction of light sources without knowing the position and the radius of the human eye. In this paper, the position and the radius of the human eye is also estimated from the image of eye. In this estimation, we assumed that the position of three light sources in the camera coordination is known. However, we do not know where the subject stands or sit in front of the imaging system. By using the estimated direction of light sources, we can estimate the surface normal and the surface reflectance of human face by photometric stereo method. We can reconstruct 3D human face by integrating the estimated surface normal in the view coordinate. The effectiveness of this method is demonstrated by experiments. Geometry model of mirrored eye Figure 1 show the geometry model of proposed imaging system. This imaging process is based on the process of mirrored ball techniques [1,2]. Since the position of human face is not restricted, the human eye will be placed at any location in a natural environment. The camera is assumed to be a pinhole camera. Therefore, we can assume the process of imaging as if a screen is placed in front the camera. The distance of camera and screen is focal length. Figure 1 shows the projection of a human face in a 3D space onto a screen by the camera in perspective. In this figure, N is the surface normal at highlight peak, L is the light source directional vector on human eye, and V is the directional vector of camera at highlight peak. D is the distance between the camera and human eye, ( ) 0 0, y x is the center of screen. The directional vector of camera at the i highlight peak ( ) i i y x , on the image can be expressed using the focal lengthα and ( ) 0 0 , y x as follow, Figure 1. Geometry of the proposed imaging system [ ] i i y y x x α − − − = , , 0 0 V . (1) The coordinates of highlight peak ( ) i i y x , can be obtained from image since the center coordinates of image is known. The focal length α of camera is also obtained by camera calibration. The directional of camera is easily calculated as is shown in Figure 1. The surface normal vector N is determined using the sphere property. Let r be the radius of eye and ( ) c c y x , be the center coordinates of eye on the image. The surface normal vector at the i highlight peak ( ) i i y x , can be calculated by