Research on Practical Sign Language Interpretation by integration of image processing of hands and other information

手部图像处理与其他信息融合的实用手语解读研究

• 批准号: 16091204
• 负责人: SHIRAI Yoshiaki
• 金额: $ 13.06万
• 依托单位: Ritsumeikan University (2005-2006)Osaka University (2004)
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research on Priority Areas
• 财政年份: 2004
• 资助国家: 日本
• 起止时间: 2004 至 2006
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-16091204/
• 关键词: sign languate interpretation; image processing; hand shape recognition; face expression measurement; information wellfare; 手指形状計測; 手話認識; 画像処理; HMM; 動画像処理; 手指形状

1. Color is used for extraction of hands and face regions. Because the color of these regions may be varied depending on persons, we developd a method of determining the skin color region in a color space. In this method, approximate regions are first extracted using an apriori skin color region. This region is refined based on the properties of extracted hands and face regions.In addition, we develop a method to avoid inclusion of the background colors while hands are moving quickly.2. When hands and face overlap, the face region is first determined in the skin region, and the rest of the skin region is searched for hands. Because the face region is not determned precisely, the certainty of the face region is used for matching hads templates in the skin region.3. We developed a method of synthesizing sign language samples from a set of real samples to increase samples for HMM learning. Although impovement of recognition rate is proved by experiments, the amount of improvement is small. One reason is that the features of synthesized samples are not suitable because of improper feature extraction. We imporved our feature extraction method.4. For recognition of sign language words, we used HMM without branching. Because the features of the same sign language word varies depending on persons, we had to prepare multiple models. We have developed a method of building HMM with branching to facilitate models of multiple sign language features.5. In order to recognize face expressions for sign language recognition, we developed a method of determining the position and shape of face parts, and detecting useful features of face face expressions. In a small scale experiment, typical expressions such ad question, joy, and anger are identified by the method.

1.颜色用于提取手部和面部区域。由于这些区域的颜色可能因人而异，因此我们开发了一种在颜色空间中确定肤色区域的方法。在该方法中，首先使用先验肤色区域提取近似区域。该区域是根据提取的手和面部区域的属性进行细化的。此外，我们开发了一种方法来避免在手快速移动时包含背景颜色。2．当手和脸部重叠时，首先在皮肤区域中确定脸部区域，然后在皮肤区域的其余部分中搜索手。由于人脸区域不能精确确定，所以利用人脸区域的确定性来匹配皮肤区域中的hads模板。 3．我们开发了一种从一组真实样本中合成手语样本的方法，以增加 HMM 学习的样本。虽然通过实验证明了识别率的提高，但是提高的幅度很小。原因之一是由于特征提取不当，导致合成样本的特征不合适。我们改进了特征提取方法。4．为了识别手语单词，我们使用不带分支的 HMM。由于同一手语单词的特征因人而异，因此我们必须准备多个模型。我们开发了一种构建带有分支的 HMM 的方法，以促进多种手语特征的模型。 5．为了识别面部表情以进行手语识别，我们开发了一种确定面部部位的位置和形状并检测面部表情的有用特征的方法。在小规模实验中，该方法识别了诸如广告问题、喜悦和愤怒等典型表情。

项目成果

動画像を用いた手指の三次元形状の推定

使用视频图像估计手指的 3D 形状

• 发表时间: 2005
• 期刊: 電子情報通信学会 HIP研究会技術研究報告集 HIP2005-58
• 作者: 松下淳一(山本弘;長谷部由起子と共著);松下 淳一;松下 淳一;神前 禎;浜田康志;松尾直志;Kaoru NAKAZONO;Tadashi Matsuo;Kaoru NAKAZONO;中園薫;C. Izumi;Atsushi Matsumono;今井章博;Kazuyuki Kanda;井上真紀;神田和幸;Kaoru NAKAZONO;Yuji NAGASHIMA;C. Izumi;Maki Inoue;Kaoru NAKAZONO;Kazunari Morimoto;K.Kawahigashi;和泉智恵;寺内美奈;神田和幸;中園薫;長嶋祐二;Eishi Watanabe;長嶋祐二;K. Morimoto;黒川隆夫;神田和幸;Kaoru NAKAZONO;Yuji NAGASHIMA;Eishi Watanabe;Yuji NAGASHIMA;K. Morimoto;K. Takao;Kazuyuki Kanda;長嶋祐二;今井章博;Eishi Watanabe;島田伸敬
• 通讯作者: 島田伸敬

Image-Based Measurement of Human Gesture and Its Applications

基于图像的人体手势测量及其应用

• 发表时间: 2007
• 期刊: IPSJ SIG Technical Reports CVIM-158
• 作者: Nobutaka Shimada;Yoshiaki Shirai
• 通讯作者: Yoshiaki Shirai

状態遷移構造の推定に基づく手話認識

基于状态转移结构估计的手语识别

• 发表时间: 2007
• 期刊: 信学技報 106・490
• 作者: 松下淳一(山本弘;長谷部由起子と共著);松下 淳一;松下 淳一;神前 禎;浜田康志;松尾直志
• 通讯作者: 松尾直志

輪郭の変形の学習による3-D手指姿勢の認識

通过学习轮廓变形进行 3D 手势识别

• 发表时间: 2005
• 期刊: 電子情報通信学会論文誌D-II (採録決定)
• 作者: 今井;島田;白井
• 通讯作者: 白井

Face, Gesture, and Action Recognition Robust Face Recognition under Various Illumination Conditions

人脸、手势和动作识别 各种光照条件下的鲁棒人脸识别

• 发表时间: 2006
• 期刊: IEICE Transactions on Information and Systems E89-D・7
• 作者: 松下淳一(山本弘;長谷部由起子と共著);松下 淳一;松下 淳一;神前 禎;浜田康志;松尾直志;Kaoru NAKAZONO;Tadashi Matsuo;Kaoru NAKAZONO;中園薫;C. Izumi;Atsushi Matsumono
• 通讯作者: Atsushi Matsumono