A Research on a Pattern Language Processing system for Human-Machine Interface

我们已经建立了一个基本的环境，通过手势来支持人与计算机之间流畅的交流。首先，我们的目标是建立一个模式语言处理系统，使其能够通过手指的语言向计算机发出指令。该系统识别摄像机图像中手指的形状和运动，并解释手指语言想要表达的指令。我们设计了一种从图像中提取手指形状的快速算法，并构建了该系统作为实验。这项实验为替代手语的新方法提供了机会，即使用棍子的方法。该方法具有处理速度快、易于指示棍子位置等优点，我们发现使用棍子的界面比使用手指语言的界面更有用，并构建了一个原型界面系统。原型界面系统从摄像机图像中提取一根棍子，得到估计的三维Coo…更具实时价值。因为很明显，操纵杆可以用作三维定点设备，所以我们构建了一个结合语音识别的多模式界面来检查可操作性。然后，我们实现了三维块状世界作为一个原型应用程序，并对其进行了评估。结果，我们发现多通道界面对于用户来说是易于使用的，同时我们进行了研究，为将来将手指语言作为模式语言使用提供了可能。为了解决在使用图像处理识别诸如手指语言之类的模式语言时的主要问题-遮挡问题，我们研究了一种使用多个摄像机互补的方法。由于语音识别对于多模式界面是必要的，我们研究了语音识别系统的配置，并开发了一种从背景噪声中提取人声的新方法。根据对该系统的评价，我们得出结论：即使在噪声环境下，该语音识别系统也是有用的。较少

We have built a basic environment which supports smooth communication between human and computers by gesture. At first, we aimed to build a pattern language processing system which make it possible to give instructions to computers by the finger's language. The system recognizes shapes and motion of fingers in a video camera image and interprets instructions which the finger's language intends to express. We designed a high speed algorithm which extracts finger shape from an image, and built the system as an experiment. This experiment afforded an opportunity for a new method substitutes for the finger language i.e.a method using a stick. This method has an advantage such as high speed processing and easiness to indicate a position of a stick.We found that an interface using a stick is more useful than the finger language, and we built a prototype interface system. The prototype interface system extracts a stick from a video camera image, and obtains the estimated three-dimensional coo … More rdinate value in real time. Because it became clear that a stick was able to be used as a three-dimensional pointing device, we built a multi-modal interface combined with speech recognition to examine operability. Then we implemented the three-dimensional block world as a prototype application and evaluated it. As a result, we found that the multi-modal interface is easy to use for users.At the same time, we carried on researches to make it possible to use the finger language as a pattern language in the future. To solve a problem of occlusion which is a main problem when we recognize pattern languages such as a finger language by using image processing, we examined a method using multi video cameras complementarily. Because speech recognition is necessary for a multi-modal interface, we examined a configuration of a speech recognition system and developed a new method which extracts human voice form background noise. According to its evaluation, we have conclude that the speech recognition system is useful even in a noisy environment. Less