Perceptual Methods for Speech Communication

言语交流可以说是最重要的社会活动。语音信号包含了语言和非语言信息。人类可以从听一段口语中学到很多东西：口语单词和语言；说话人的身份、性别、口音、年龄、语言水平、情绪状态、健康状况；基于捕捉到的环境声音和噪声的背景环境；以及任何携带该信号的通信/记录媒体，基于该媒体在该信号上所印的可感知的“特征”。此外，即使在语音信号严重损坏的情况下，人类听者也能很好地理解嵌入的语言和副语言信息。在电子设备上运行的信号处理算法中体现的计算智能（目前）还不能与人类的表现相匹敌。然而，新兴的万物互联将进一步增加人类与电子设备之间以及以网络设备为媒介的人类之间的语音交流机会。这种语音交流可以随时随地发生。因此，接收退化语音信号的设备将迅速增加。为了解决这一挑战，我们建议研究语音信号处理方法，使语音信息的机器提取能够适应声音退化，即环境噪声和混响。创新的想法将借鉴语音认知科学和鲁棒信号建模和恢复技术的新进展。受人类听觉语音信号皮层处理的启发，新的语音信号处理方法将得到发展。此外，在今天的移动电话中无处不在的语音信号模型将得到改进。一个主要目标是提高当前机器对嵌入在退化语音信号中的口头信息的理解水平，并通过这样做为人类听众提供高度可理解的语音。新的知识、方法和算法将产生新的能力，以增强现有和未来的语音通信和声学接口技术。除了培养高素质的研发人员外，这项研究还将产生新的更好的工具，使工业界和学术界都受益。其中一种工具将使研发工程师能够优化其语音增强算法。将从拟议的研究中受益的行业部门包括通信和信息技术、游戏、机器中介学习、助听器、健康和健身以及机器人。新的知识和解决方案也有望为室内和室外环境的声景设计等其他领域做出贡献。

Speech communication is arguably the most important social activity. Speech signals are packed with verbal and non-verbal information. Humans can learn a great deal from listening to just a snippet of spoken speech: the spoken words and language; the speaker’s identity, gender, accent, age, language proficiency, emotional state, health; the background environment based on the ambient sounds and noises captured; and any communications/recording media which carried the signal, based on perceivable “signatures” imprinted by the media on the signal. Moreover, human listeners are remarkably capable of comprehending the embedded linguistic and paralinguistic information even when the speech signal is highly corrupted. Computational intelligence, as embodied in the signal processing algorithms that run on electronic devices, is not (yet) a match for human performance. Nevertheless, the emerging Internet of Everything will further multiply the opportunities for speech communication between humans and electronic devices and between humans mediated by networked devices. Such speech communication can happen anytime and anywhere. As a result, the instances of devices picking up degraded speech signals will mushroom. To address this challenge, we propose to research speech signal processing methods that would enable machine extraction of speech information to be resilient to acoustic degradations, i.e., ambient noise and reverberation. Ideas for innovation will draw upon emerging advances in speech cognitive science and robust signal modeling and recovery techniques. New speech signal processing methods inspired by human cortical processing of acoustic speech signals will be developed. Moreover, a speech signal model ubiquitously deployed in today’s cellular telephones will be revamped. A major goal is to advance the current level of machine comprehension of verbal information embedded in degraded speech signals, and by doing so also provide human listeners with highly intelligible speech. The new knowledge, methods, and algorithms anticipated from the proposed research will engender new capabilities for enhancing existing and enabling future speech communication and acoustic interface technologies. Besides training highly qualified R&D personnel, the research will produce new and better tools to benefit both industry and academia. One type of tool to be produced will enable R&D engineers to optimize their speech enhancement algorithms. Industry sectors that will benefit from the proposed research include communications and information technology, gaming, machine-mediated learning, hearing instruments, health and fitness, and robotics. The new knowledge and solutions are anticipated to also contribute to farther fields such as soundscape design for indoor and outdoor environments.