Research on the word recognition method based on voice and lip shape movements in very noisy circumstances

点击翻译按钮获取中文摘要

Word recognition techniques have been investigated by many researchers. In these researches, many kinds of characteristic parameters of speech signal are studied and many kinds of recognition methods are proposed. The Cepstrum parameter and HMM (Hidden Markov Model) are representative examples of them.In the calm circumstances, they enable us to recognize natural speed speaking speech with relatively high recognition rate. However, in the noisy circumstances, it is still difficult to achieve the high word recognition rate by using methods that depend on only auditory information.On the other hand, it is well known that human has ability to understand other person's talks just by watching his mouth movements without any auditory information. This ability is called "Lip reading". If the word recognition system based on voices and lip shape movements, it can be expected to offer not only effective means of word recognition in very noisy circumstances but also easy human-machine communication methods.To construct the system described above, following problems must be solved.1. Very fast and exact extraction of lip shape from series of face images is required.2. Parameters that describe lip shapes must be investigated.3. Accurate recognition method is required.Because of these problems, the word recognition systems which uses lip shape movements have not been developed until now. In this research, we had tried to develop the real time word recognition system based on the voices and lip shape movements on the commercial base personal computer. For realization of fast and exact operation, the technique called as the modified Sampled Active Contour Model (modified SACM) is adopted to extract lip shapes. For describing the extracted lip shapes, new parameter is proposed in this paper. And the recognition of the lip shape movements is achieved by HMM according to the proposed new parameters.