Development of high-definition audio-visual speech communication systems based on the knowledge of /kansei/ information processing

传递说话者的思想和情感对于开发先进的通信系统是非常重要的。由于不仅语音信号本身，说话人面部的运动图像也包含着丰富的信息，以便理解说话人想说什么，所以这些信息必须准确地传递给听话人。然而，从视听语音信号中提取的物理参数与/感性/信息之间的关系尚不清楚，本研究的最终目标是开发高清晰度的视听语音通信系统。为此，研究了说话者人脸运动图像对语音理解的影响。心理物理实验结果表明，嘴唇周围区域对于理解说话者所说的话是至关重要的。此外，来自视听语音信号的/感性/印象对于语音理解也很重要。

It is important to transmit talker's mind including emotion for developing advanced communication system. Because not only speech signal itself but also moving image of talkers face includes rich information to understand what talker wants to tell, these information should be transmit accurately to the listeners. However, the relationship between the physical parameter extracted from audio-visual speech signal and /kansei/ information is unclear.The final goal of this study is to develop high-definition audio-visual speech communication systems. To do this, the effect of moving image of talker's face to speech understanding was investigated. The results of psychophysical experiments indicate that around the lips area is crucial to understand what talker says. Moreover, /kansei/ impression from audio-visual speech signal is also important for speech understanding.