A system for cocktail party effect using large sensor array

点击翻译按钮获取中文摘要

A psychological term "cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this technological research, the cocktail party effect is focused on in order to construct a new type of sensing system.An audio-visual sensor fusion system which integrates audio signals and sequential images on time-space coordinates, is proposed in this research. Furthermore, a large microphone array system is incorporated into the proposed system. Summaries of the research are as follows.A prototype system with the large microphone array of 156 microphones has been constructed. DSP modules for parallel processing have been designed and assembled. Communication modules for DSP-to-DSP communications are also designed and assembled. Furthermore, software tools including an automatic coding software for communication tasks and debugger have been constructed. Using these hardware and software, real-time sound source separation can be demonstrated.Theoretical analysis for the signal separation using the large microphone array are performed. A new signal separation method are proposed, which is a combination of sub-space method and grouping method based on the theory of independent component analysis.Results of experiments using the prototype system demonstrated the advantages of the large microphone array and audio-visual sensor fusion scheme.