Microphone Arrays for Immersive Voice Communications

沉浸式语音通信可以有效地促进小组协作和远程会议

Immersive voice communication facilitates group collaborations and teleconferencing efficiently at low costs. It is becoming an integral part of modern communication networks. A key technical challenge for an immersive voice communication system is the ability to acquire high-fidelity acoustic, audio, and speech signals while keeping spatial information of the sources intact so that it is possible for the remote listener to sense the acoustic environment, follow a panel of talkers, and distinguish them by listening to the reproduction of the signals. To acquire high-fidelity speech and preserve the sound realism, it is necessary to use microphone arrays. Although microphone array processing has been studied for a few decades, existing techniques produce frequency-dependent responses, which may cause much distortion to the signal of interest, noise, interferences, and sound realism; as a result, they are not sufficient for immersive communication systems. This proposal is therefore organized to address the technical issues of microphone array design and beamforming and develop theory, methods, and technologies that can allow acquisition, processing, and reproduction of high-fidelity spatial sound for immersive voice communication systems. This proposal contains the following key technical innovations and merits. 1) A fundamental formulation of and a signal processing theory for frequency-invariant microphone array design and beamforming for signals with frequency range from 60 Hz to 20 kHz. 2) A new generation of technologies for linear microphone array design and frequency-invariant and robust beamforming. 3) A new generation of circular and spherical microphone array design and frequency-invariant and robust beamforming algorithms. 4) A signal processing theory for optimization of microphone array geometry with some given performance criteria. 5) A signal processing theory and a new generation of technologies for distributed microphone array processing. 6) A practical system that takes ubiquitous acoustic signals as input and produces high-fidelity output signals with source realism and identifiable source identities. It is clear that the proposed research is in the heart of immersive voice communication. The resulting technologies can have a broad range of applications besides immersive voice communication including telecollaboration, human-machine interfaces, acoustic surveillance, security, smartphones, smart home systems, tablet PCs, hearing aids, etc. As the demand of microphone arrays is increasing, the research result of this project will have undoubtedly a profound impact on the society and it will be truly indispensable.