Study on Speech Recognition by Uniting Image Sequence and Acoustic Signal

点击翻译按钮获取中文摘要

(1) Development of method for recovering 3-D lips shape from image sequence(2) We developed a method for extracting the absolute depth information from the depth information having an indefinite scale, which is obtained by a moire imaging, with a simple reflection model for an object surface. It was confirmed through simulations and real image experiments that by considering not only a diffuse reflectance but also a specular one, an unbiased depth can be computed.(3) Development of vision-based speech recognition method using 3-D lips shapeWe proposed a method of a vowel recognition and a word recognition using 3-D lips shape information. The advantage of the proposed method is that an invariant recognition against a relative angle between a face and a camera can be done and a recognition rate purely improves by applying the proposed method. The above advantage was confirmed by real image experiments.Development of lips image generation method based on multiple light sourceBy extending the method developed in (1) to a multiple light source system, we proposed a method for computing the reflectances of both a lips region and a skin region. As a result, it was shown that lips images from an arbitral vewpoint can be generated.