Anthropomorphic and efficient multimedia communication

点击翻译按钮获取中文摘要

The aim of our research is to devise means to build machines that perform more like humans in interacting with humans and to perform more efficiently in communicating human-targeted media content. A human listening to another person's speech can recognize more than just the spoken words. From the speech signal alone, the listener can instinctively extract paralinguistic information such as the speaker's accent, gender, age, identity, emotion, and even health and mental state. Humans can also instinctively judge the quality of the speech signal. We aim to develop speech processing algorithms that will make machines better able to recognize the emotion in speech signals and to judge how intelligible is a given speech signal. The former will make machines better able to understand human speech and the latter will equip machines with a capability to improve the quality of distorted speech signals. We also propose to devise methods to enable machines to make human-like judgments on the visual quality of a given video signal. Lastly, we propose to research methods to enable signal sensors to communicate without the aid of a network infrastructure. Advanced coding schemes will be developed to provide error resilient communication while conserving energy. The proposed research program will advance machine intelligence and communication technologies that can be used, for instance, to build energy-efficient sensors that make up a "smart" wearable audio-visual communication system. Such a system can integrate the functions of wireless headsets, hearing aids, microphones, speech recognizers, and cameras. Both environmentally sensed and electronically received audio and visual signals can be processed to optimize the audio-visual experience of the wearer and/or remote recipients. For instance, audio signals can be processed to ameliorate effects of acoustic interferences and to best suit the hearing profile of each ear. Audio-visual signals can be processed for the best delivered user experience commensurate with the communication resources available, such as battery and communication channel capacity. The built-in speech recognizer can orchestrate services with a human touch by reacting to speech affects.