A system for cocktail party effect using large sensor array

心理学术语“鸡尾酒会效应”指的是从其他环境噪音中提取重要信号的生物特殊能力。在这项技术研究中，为了构建一种新型的传感系统，重点研究了鸡尾酒会效应，提出了一种在时空坐标上融合音频信号和序列图像的视听传感器融合系统。此外，在所提出的系统中加入了大型麦克风阵列系统。主要研究内容如下：搭建了一个由156个传声器组成的大型传声器阵列的原型系统。设计并组装了用于并行处理的数字信号处理器模块。设计并组装了数字信号处理器与数字信号处理器的通信模块。此外，还构建了包括通信任务自动编码软件和调试器在内的软件工具。利用这些硬件和软件实现了实时的声源分离，并对大型麦克风阵列信号分离进行了理论分析。提出了一种基于独立分量分析理论的子空间方法和分组方法相结合的信号分离方法，并利用原型系统进行了实验验证，验证了大麦克风阵列与声光传感器融合方案的优越性。

A psychological term "cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this technological research, the cocktail party effect is focused on in order to construct a new type of sensing system.An audio-visual sensor fusion system which integrates audio signals and sequential images on time-space coordinates, is proposed in this research. Furthermore, a large microphone array system is incorporated into the proposed system. Summaries of the research are as follows.A prototype system with the large microphone array of 156 microphones has been constructed. DSP modules for parallel processing have been designed and assembled. Communication modules for DSP-to-DSP communications are also designed and assembled. Furthermore, software tools including an automatic coding software for communication tasks and debugger have been constructed. Using these hardware and software, real-time sound source separation can be demonstrated.Theoretical analysis for the signal separation using the large microphone array are performed. A new signal separation method are proposed, which is a combination of sub-space method and grouping method based on the theory of independent component analysis.Results of experiments using the prototype system demonstrated the advantages of the large microphone array and audio-visual sensor fusion scheme.