Japanese Machine Lip-reading System Based on Human Lip-reading Method

日本基于人类唇读方法的机器唇读系统

• 批准号: 23700672
• 负责人: MIYAZAKI Tsuyoshi
• 金额: $ 1.91万
• 依托单位: Kanagawa Institute of Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Young Scientists (B)
• 财政年份: 2011
• 资助国家: 日本
• 起止时间: 2011 至 2013
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-23700672/
• 关键词: 機械読唇; 障害者支援; 画像処理; 画像認識; パターン認識; 聴覚障害者支援; 発話映像; 口形画像; 口形検出; 動画像認識; 聴覚障がい者支援

I have improved the detection rate of mouth shapes from Japanese utterance images, and proposed a recognition method of Japanese words. I aimed for more than 90% at the detection rate of mouth shapes, but it was finally with 82.7% by using the optical flow and the template matching. This shows that the detection rate was improved by 8.6% from the grant start. By the recognition of the Japanese utterance word, I proposed the calculation method as reliability in consideration of the length of utterance word, and 36 prefectures were recognized from 47 prefectures utterance images by the experiment. The recognition rate became approximately 76.6%. In future, I consider a method to improve accuracy to extract the period that the mouth shapes are formed.

提高了日语语音图像中口型的检测率，提出了一种日语单词的识别方法。我的目标是在90%以上的嘴型检测率，但它最终与82.7%，使用光流和模板匹配。这表明，自拨款开始以来，破案率提高了8.6%。通过对日语发音词的识别，提出了考虑发音词长度的信度计算方法，并通过实验从47个都道府县的发音图像中识别出36个都道府县。识别率约为76.6%。在未来，我考虑一种方法，以提高准确性，提取嘴的形状形成的时期。

项目成果

Analysis of Mouth Shape Deformation Rate for Generation of Japanese Utterance Images Automatically

自动生成日语语音图像的口形变形率分析

• 发表时间: 2014
• 作者: Tsuyoshi Miyazaki;Toyoshiro Nakashima;Naohiro Ishii;Tsuyoshi Miyazaki and Toyoshiro Nakashima
• 通讯作者: Tsuyoshi Miyazaki and Toyoshiro Nakashima

Mouth Shape Detection Based on Template Matching and Optical Flow for Machine Lip Reading

基于模板匹配和光流的嘴形检测机器唇读

• 发表时间: 2013
• 期刊: International Journal of Software Innovation
• 影响因子: 0.6
• 作者: Tsuyoshi Miyazaki;Toyoshiro Nakashima;Naohiro Ishii
• 通讯作者: Naohiro Ishii

日本語発話時の口形変化量の分析と発話映像自動生成への適用

日语语音时嘴型变化分析及其在语音视频自动生成中的应用

• 发表时间: 2013
• 作者: 宮崎剛;中島豊四郎
• 通讯作者: 中島豊四郎

Producing text and speech from video images of lips movement photographed in speaking Japanese by using mouth shape sequence code : an experimental system to communicate with hearing impaired persons

使用口形序列代码从日语中拍摄的嘴唇运动视频图像生成文本和语音：与听力障碍者交流的实验系统

• 发表时间: 2012
• 作者: Shiori KAWAHATA;Eiko KOYAMA;Tsuyoshi MIYAZAKI and Fujio YAMAMOTO
• 通讯作者: Tsuyoshi MIYAZAKI and Fujio YAMAMOTO

口形ベースの機械読唇における単語認識手法の提案と評価

基于口型的机器唇读词语识别方法的提出与评价

• 发表时间: 2014
• 作者: 宮崎剛;中島豊四郎
• 通讯作者: 中島豊四郎