Development of high-definition audio-visual speech communication systems based on the knowledge of /kansei/ information processing

如何传递说话人的思想和情感，对于发展先进的通信系统有着重要的意义。由于语音信号本身以及说话人面部的运动图像都包含着丰富的信息，需要准确地将这些信息传递给听者。然而，从视听语音信号中提取的物理参数与/感性/信息之间的关系尚不清楚，本研究的最终目标是开发高清晰度的视听语音通信系统。为了做到这一点，说话人的脸的运动图像的影响，语音理解进行了研究。心理物理学实验的结果表明，嘴唇周围的区域对理解说话者所说的话至关重要。此外，视听言语信号中的/感性/印象对于言语理解也很重要。

It is important to transmit talker's mind including emotion for developing advanced communication system. Because not only speech signal itself but also moving image of talkers face includes rich information to understand what talker wants to tell, these information should be transmit accurately to the listeners. However, the relationship between the physical parameter extracted from audio-visual speech signal and /kansei/ information is unclear.The final goal of this study is to develop high-definition audio-visual speech communication systems. To do this, the effect of moving image of talker's face to speech understanding was investigated. The results of psychophysical experiments indicate that around the lips area is crucial to understand what talker says. Moreover, /kansei/ impression from audio-visual speech signal is also important for speech understanding.