Anthropomorphic and efficient multimedia communication

我们研究的目的是设计出一种方法，使机器在与人类互动时表现得更像人类，并在交流针对人类的媒体内容时表现得更有效。一个人在听另一个人说话时，能识别的不仅仅是说出来的单词。仅从语音信号中，听者就能本能地提取副语言信息，如说话者的口音、性别、年龄、身份、情绪，甚至健康和精神状态。人类也可以本能地判断语音信号的质量。我们的目标是开发语音处理算法，使机器能够更好地识别语音信号中的情感，并判断给定语音信号的可理解程度。前者将使机器能够更好地理解人类的语言，后者将使机器具备改善扭曲语音信号质量的能力。我们还建议设计方法，使机器能够对给定视频信号的视觉质量做出类似人类的判断。最后，我们建议研究使信号传感器无需网络基础设施的帮助即可进行通信的方法。将开发先进的编码方案，在节约能源的同时提供抗错误通信。拟议中的研究项目将推动机器智能和通信技术的发展，例如，构建节能传感器，构成“智能”可穿戴视听通信系统。该系统可以集成无线耳机、助听器、麦克风、语音识别器和摄像头的功能。环境感测和电子接收的音频和视觉信号都可以被处理以优化佩戴者和/或远程接收者的视听体验。例如，可以对音频信号进行处理，以改善声学干扰的影响，并最适合每只耳朵的听力状况。可以根据可用的通信资源（如电池和通信信道容量）对视听信号进行处理，以获得最佳的用户体验。内置的语音识别器可以通过对语音影响做出反应来协调服务。

The aim of our research is to devise means to build machines that perform more like humans in interacting with humans and to perform more efficiently in communicating human-targeted media content. A human listening to another person's speech can recognize more than just the spoken words. From the speech signal alone, the listener can instinctively extract paralinguistic information such as the speaker's accent, gender, age, identity, emotion, and even health and mental state. Humans can also instinctively judge the quality of the speech signal. We aim to develop speech processing algorithms that will make machines better able to recognize the emotion in speech signals and to judge how intelligible is a given speech signal. The former will make machines better able to understand human speech and the latter will equip machines with a capability to improve the quality of distorted speech signals. We also propose to devise methods to enable machines to make human-like judgments on the visual quality of a given video signal. Lastly, we propose to research methods to enable signal sensors to communicate without the aid of a network infrastructure. Advanced coding schemes will be developed to provide error resilient communication while conserving energy. The proposed research program will advance machine intelligence and communication technologies that can be used, for instance, to build energy-efficient sensors that make up a "smart" wearable audio-visual communication system. Such a system can integrate the functions of wireless headsets, hearing aids, microphones, speech recognizers, and cameras. Both environmentally sensed and electronically received audio and visual signals can be processed to optimize the audio-visual experience of the wearer and/or remote recipients. For instance, audio signals can be processed to ameliorate effects of acoustic interferences and to best suit the hearing profile of each ear. Audio-visual signals can be processed for the best delivered user experience commensurate with the communication resources available, such as battery and communication channel capacity. The built-in speech recognizer can orchestrate services with a human touch by reacting to speech affects.