A Research on a Pattern Language Processing system for Human-Machine Interface

人机界面模式语言处理系统的研究

• 批准号: 06808039
• 负责人: YOSHIDA Takaichi
• 金额: $ 1.22万
• 依托单位: Kyushu Institute of Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for General Scientific Research (C)
• 财政年份: 1994
• 资助国家: 日本
• 起止时间: 1994 至 1995
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-06808039/
• 关键词: human interface; multi-modal interface; information integration; image processing; image recognition; speech recognition; speech extraction; ヒューマンインターフェース; 並列分散処理

We have built a basic environment which supports smooth communication between human and computers by gesture. At first, we aimed to build a pattern language processing system which make it possible to give instructions to computers by the finger's language. The system recognizes shapes and motion of fingers in a video camera image and interprets instructions which the finger's language intends to express. We designed a high speed algorithm which extracts finger shape from an image, and built the system as an experiment. This experiment afforded an opportunity for a new method substitutes for the finger language i.e.a method using a stick. This method has an advantage such as high speed processing and easiness to indicate a position of a stick.We found that an interface using a stick is more useful than the finger language, and we built a prototype interface system. The prototype interface system extracts a stick from a video camera image, and obtains the estimated three-dimensional coo … More rdinate value in real time. Because it became clear that a stick was able to be used as a three-dimensional pointing device, we built a multi-modal interface combined with speech recognition to examine operability. Then we implemented the three-dimensional block world as a prototype application and evaluated it. As a result, we found that the multi-modal interface is easy to use for users.At the same time, we carried on researches to make it possible to use the finger language as a pattern language in the future. To solve a problem of occlusion which is a main problem when we recognize pattern languages such as a finger language by using image processing, we examined a method using multi video cameras complementarily. Because speech recognition is necessary for a multi-modal interface, we examined a configuration of a speech recognition system and developed a new method which extracts human voice form background noise. According to its evaluation, we have conclude that the speech recognition system is useful even in a noisy environment. Less

我们已经建立了一个基本的环境，支持人机之间通过手势流畅的交流。首先，我们的目标是建立一个模式语言处理系统，使通过手指语言向计算机发出指令成为可能。该系统可以识别摄像机图像中手指的形状和运动，并解释手指语言想要表达的指令。设计了一种从图像中提取手指形状的高速算法，并搭建了该系统作为实验。这个实验为用一种新的方法代替手指语言提供了机会，即使用一根棍子的方法。该方法具有加工速度快、易于指示操纵杆位置等优点。我们发现使用摇杆的界面比手指语言更有用，于是我们建立了一个原型界面系统。该原型界面系统从摄像机图像中提取一根棍子，并实时得到估计的三维坐标值。因为很明显，一根棍子可以用作三维指向设备，我们建立了一个多模态界面，结合语音识别来检查可操作性。然后我们实现了三维块世界作为原型应用，并对其进行了评价。结果，我们发现多模态界面对用户来说很容易使用。同时，我们也进行了一些研究，希望将来能够将手指语言作为一种模式语言。为了解决图像处理识别模式语言（如手指语言）时的主要问题——遮挡问题，我们研究了一种多摄像机互补的方法。由于语音识别是多模态界面的必要条件，我们研究了语音识别系统的结构，并开发了一种从背景噪声中提取人声的新方法。通过对该系统的评估，我们得出结论，即使在嘈杂的环境中，该语音识别系统也是有用的。少

项目成果

Y.Ito,T.Ohashi and T.Ejima: "Considerations on Designing a Decision-Tree with Multiple Features" Fourth International Workshop on Frontiers of Handwriting Recoginition. 362-369 (1994)

Y.Ito、T.Ohashi 和 T.Ejima：“Considerations on Designing a Decision-Tree with Multiple Features”第四届手写识别前沿国际研讨会。

T.Ohashi: "Multimodal interface with Speech and Motion of Stick : CoSMoS" Symbiosis of Human and Artifact,Auzai et al. (Editors), Elsevier. 20A. 207-212 (1995)

T.Ohashi：“带有语音和棍棒运动的多模态界面：CoSMoS”人类与人工制品的共生，Auzai 等人。

Nobufumi Ono, Atsushi Matsunaga, Takeshi Ohashi and Toshiaki Ejima: ""Gesture recognition to command motions in three dimensional space" (in Japanese)" Proc.of Meeting of Image Recognition and Understanding, MIRU'94. Vol.II. 183-190 (1994)

Nobufumi Ono、Atsushi Matsunaga、Takeshi Ohashi 和 Toshiaki Ejima：““在三维空间中指挥运动的手势识别”（日语）”Proc.of Meeting of Image Recognition and Understanding，MIRU94。

山之内毅: "指示棒と音声を用いたコミュニケーション環境CoSMoSの音声認識処理の検討" 電子情報通信学会技術研究会報告. PRU94-81. 71-78 (1994)

Takeshi Yamanouchi：“CoSMoS 中的语音识别处理研究，使用指点杆和语音的通信环境” IEICE 技术研究组报告。

Takeshi Yamanouchi, Takeshi Ohashi and Toshiaki Ejima: ""A Design of CoSMoS witch is the communication environment with speech and motion of stick" (in Japanese)" Technical report of IEICE. PRU94-81. 71-78 (1994)

Takeshi Yamanouchi、Takeshi Ohashi 和 Toshiaki Ejima：“CoSMoS 的设计是通过语音和棍子运动进行通信的环境”（日语）”IEICE 的技术报告。