Study on Speech Recognition by Uniting Image Sequence and Acoustic Signal

图像序列与声音信号相结合的语音识别研究

• 批准号: 11680397
• 负责人: TAGAWA Norio
• 金额: $ 2.43万
• 依托单位: Tokyo Metropolitan University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 1999
• 资助国家: 日本
• 起止时间: 1999 至 2000
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-11680397/
• 关键词: Speech recognition; 3-D lips shape; vowel recognition; word recognition; HMM; moire analysis; reflectance; region segmentation; 領域分離; 発語認識; 口形状; 3次元形状; モアレトポグラフィ

(1) Development of method for recovering 3-D lips shape from image sequence(2) We developed a method for extracting the absolute depth information from the depth information having an indefinite scale, which is obtained by a moire imaging, with a simple reflection model for an object surface. It was confirmed through simulations and real image experiments that by considering not only a diffuse reflectance but also a specular one, an unbiased depth can be computed.(3) Development of vision-based speech recognition method using 3-D lips shapeWe proposed a method of a vowel recognition and a word recognition using 3-D lips shape information. The advantage of the proposed method is that an invariant recognition against a relative angle between a face and a camera can be done and a recognition rate purely improves by applying the proposed method. The above advantage was confirmed by real image experiments.Development of lips image generation method based on multiple light sourceBy extending the method developed in (1) to a multiple light source system, we proposed a method for computing the reflectances of both a lips region and a skin region. As a result, it was shown that lips images from an arbitral vewpoint can be generated.

(1)从图像序列中恢复三维唇形的方法研究(2)提出了一种从云纹成像获得的具有不定尺度的深度信息中提取绝对深度信息的方法，该方法具有简单的物体表面反射模型。通过仿真和实像实验证实，既考虑漫反射，又考虑镜面反射，可以计算出无偏深度。(3)基于三维唇形的视觉语音识别方法的发展我们提出了一种基于三维唇形信息的元音识别和单词识别方法。该方法的优点是可以对人脸与相机之间的相对角度进行不变识别，并且可以提高识别率。通过实像实验验证了上述优点。基于多光源的嘴唇图像生成方法的发展通过将(1)中开发的方法扩展到多光源系统，我们提出了一种计算嘴唇区域和皮肤区域反射率的方法。结果表明，可以从任意点生成嘴唇图像。

项目成果

K.UDA, N.TAGAWA, A.MINAGAWA, T.MORIYA: "Efectiveness evaluation of speech feature including 3-D information for lipreading (in Japanese)"Proc. Of the 2000 Society Conference of IEICE. No.D-12-54. 241 (2000)

K.UDA、N.TAGAWA、A.MINAGAWA、T.MORIYA：“包括唇读 3D 信息在内的语音特征的有效性评估（日语）”Proc。

K.NIIKURA, A.MINAGAWA, N.TAGAWA, T.MORITA: "Determination of the absolute number of moirefringes using shading in formation"Proc.of Advances in Pattern Recoguition and Digital Techniques. 154-159 (1999)

K.NIIKURA、A.MINAGAWA、N.TAGAWA、T.MORITA：“使用形成中的阴影确定莫尔条纹的绝对数量”Proc.of Advances in Pattern Recoguition and Digital Techniques。

浅倉, 皆川, 田川, 守屋: "陰影情報を用いたモアレ縞の絶対縞次数決定法"電子情報通信学会技術研究報告. IE-99-36. 81-88 (1999)

Asakura、Minakawa、Takawa、Moriya：“使用阴影信息的莫尔条纹的绝对条纹顺序确定方法”IEICE 技术报告。

S.NAGANUMA, A.MINAGAWA, N.TAGAWA, T.MORIYA: "Computation of 3-D shape and reflectance with shading model and moire phase (in Japanese)"Proc. Of the 2001 IEICE General Conference. No.D-12-67. 234 (2001)

S.NAGANUMA、A.MINAGAWA、N.TAGAWA、T.MORIYA：“使用阴影模型和莫尔相位计算 3-D 形状和反射率（日语）”Proc。

稲垣, 田川, 皆川, 守屋: "モアレ位相と光の反射モデルに基づく物体の3次元形状と反射率の計算"電子情報通信学会技術研究報告. PRMU2000-229. 51-56 (2001)

Inagaki、Takawa、Minakawa、Moriya：“基于莫尔相位和光反射模型的物体的 3D 形状和反射率计算”PRMU2000-229 (2001)。