How, What and Who in Human Communication: Movement of Face and Voice

这个项目的目的是利用信号处理、生物学和行为学技术相结合的方法，从人脸和语音中识别音调、情感和身份的基本特征，以开发一个全面的视听语音处理和交流模型。这项研究将显著提高对声调语言和情感交流中听觉-视觉知觉和产生的基础的理解，促进神经生理过程和听觉-视觉语音处理之间的联系，并有助于在自动人物识别、自动语音识别、文本到语音系统和助听者说话头部辅助方面的应用。

The aim of this project is to identify the essential characteristics of tone, affect, and identity from face and voice using a combination of signal processing, biological, and behavioural techniques in order to develop a comprehensive model of auditory-visual speech processing and communication. This research will significantly improve understanding of the basis of auditory-visual perception and production in tonal languages and in affective communication, facilitate links between neurophysiological processes and auditory-visual speech processing; and contribute to applications in automatic person recognition, automatic speech recognition, text-to-speech systems, and talking head aids for the hearing impaired.