Development of human machine interface by using lip reading technique in noisy circumstances

近年来，人们逐渐认识到根据人的指令进行辅助行为而不是单独的行为的重要性。在这种情况下，应该考虑人与机器人之间的通信方式。单词识别技术已经在几个研究人员中得到了开发，在平静的情况下，它可以用于实际应用。但在嘈杂的情况下，它们仍然不适用，有许多问题需要解决。对于噪声环境下的单词识别方法，本文主要研究了唇读方法。本发明不需要用户额外的训练来控制机器人，本研究将人类说话的视频图像输入到系统中，并自动提取唇形动作，为文字识别提供参数。针对这些问题，采用活动轮廓模型技术的硬件实现，实现了NTSC视频信号速率唇形的自动提取。在以后的工作中，保留了复杂背景下唇区分离技术的发展方向。

The importance of assistant behavior according to the human commands instead of the stand-alone behavior has been recognized in recent years. In such cases, communication methods between human and robots should be considered. The word recognition technique had been developed in several researchers and in calm situation it can be used for practical use. But in noisy circumstances, they are not still applicable and have a number of problems to solve. For word recognition method in noisy circumstances, the lip reading method is investigated in our research. By the lip reading method, additional trainings of user are not required to control robots.In this research, the video images of human speaking are input to the system and the lip shape movements are automatically extracted to provide word recognition parameters. For these automatic extractions of lip shapes of the NTSC video signal rate are realized by hardware realization of the active contour model techniques. For future works, the development of the lip area separation technique from complex background is remained.