Study on Speech Recognition by Uniting Image Sequence and Acoustic Signal

(1)从图像序列中恢复三维唇形的方法研究(2)提出了一种从云纹成像获得的具有不定尺度的深度信息中提取绝对深度信息的方法，该方法具有简单的物体表面反射模型。通过仿真和实像实验证实，既考虑漫反射，又考虑镜面反射，可以计算出无偏深度。(3)基于三维唇形的视觉语音识别方法的发展我们提出了一种基于三维唇形信息的元音识别和单词识别方法。该方法的优点是可以对人脸与相机之间的相对角度进行不变识别，并且可以提高识别率。通过实像实验验证了上述优点。基于多光源的嘴唇图像生成方法的发展通过将(1)中开发的方法扩展到多光源系统，我们提出了一种计算嘴唇区域和皮肤区域反射率的方法。结果表明，可以从任意点生成嘴唇图像。

(1) Development of method for recovering 3-D lips shape from image sequence(2) We developed a method for extracting the absolute depth information from the depth information having an indefinite scale, which is obtained by a moire imaging, with a simple reflection model for an object surface. It was confirmed through simulations and real image experiments that by considering not only a diffuse reflectance but also a specular one, an unbiased depth can be computed.(3) Development of vision-based speech recognition method using 3-D lips shapeWe proposed a method of a vowel recognition and a word recognition using 3-D lips shape information. The advantage of the proposed method is that an invariant recognition against a relative angle between a face and a camera can be done and a recognition rate purely improves by applying the proposed method. The above advantage was confirmed by real image experiments.Development of lips image generation method based on multiple light sourceBy extending the method developed in (1) to a multiple light source system, we proposed a method for computing the reflectances of both a lips region and a skin region. As a result, it was shown that lips images from an arbitral vewpoint can be generated.