Anthropomorphic and efficient multimedia communication

我们研究的目的是设计方法来构建机器，使其在与人类互动时表现得更像人类，并在传达针对人类的媒体内容时表现得更有效。一个人听另一个人的讲话可以识别的不仅仅是所说的话。仅从语音信号中，听者就可以本能地提取出说话者的口音、性别、年龄、身份、情感，甚至健康和精神状态等非语言信息。人类也可以本能地判断语音信号的质量。我们的目标是开发语音处理算法，使机器能够更好地识别语音信号中的情感，并判断如何理解是一个给定的语音信号。前者将使机器能够更好地理解人类语音，后者将使机器具有改善失真语音信号质量的能力。我们还建议设计方法，使机器能够对给定视频信号的视觉质量做出类似人类的判断。最后，我们建议研究方法，使信号传感器进行通信，而无需网络基础设施的帮助。将开发先进的编码方案，以提供容错通信，同时节省能源。拟议的研究计划将推进机器智能和通信技术，例如，可用于构建构成“智能”可穿戴视听通信系统的节能传感器。这样的系统可以集成无线耳机、助听器、麦克风、语音识别器和相机的功能。环境感测的和电子接收的音频和视觉信号都可以被处理以优化佩戴者和/或远程接收者的视听体验。例如，可以处理音频信号以改善声学干扰的影响，并最好地适合每只耳朵的听力曲线。可以处理视听信号，以获得与可用的通信资源（例如电池和通信信道容量）相称的最佳用户体验。内置的语音识别器可以通过对语音影响做出反应来协调具有人性化的服务。

The aim of our research is to devise means to build machines that perform more like humans in interacting with humans and to perform more efficiently in communicating human-targeted media content. A human listening to another person's speech can recognize more than just the spoken words. From the speech signal alone, the listener can instinctively extract paralinguistic information such as the speaker's accent, gender, age, identity, emotion, and even health and mental state. Humans can also instinctively judge the quality of the speech signal. We aim to develop speech processing algorithms that will make machines better able to recognize the emotion in speech signals and to judge how intelligible is a given speech signal. The former will make machines better able to understand human speech and the latter will equip machines with a capability to improve the quality of distorted speech signals. We also propose to devise methods to enable machines to make human-like judgments on the visual quality of a given video signal. Lastly, we propose to research methods to enable signal sensors to communicate without the aid of a network infrastructure. Advanced coding schemes will be developed to provide error resilient communication while conserving energy. The proposed research program will advance machine intelligence and communication technologies that can be used, for instance, to build energy-efficient sensors that make up a "smart" wearable audio-visual communication system. Such a system can integrate the functions of wireless headsets, hearing aids, microphones, speech recognizers, and cameras. Both environmentally sensed and electronically received audio and visual signals can be processed to optimize the audio-visual experience of the wearer and/or remote recipients. For instance, audio signals can be processed to ameliorate effects of acoustic interferences and to best suit the hearing profile of each ear. Audio-visual signals can be processed for the best delivered user experience commensurate with the communication resources available, such as battery and communication channel capacity. The built-in speech recognizer can orchestrate services with a human touch by reacting to speech affects.