Three dimensional cocktail party effect processor using audio-visual sensor fusion technique

心理学术语“鸡尾酒会效应”是指从其他环境噪音中提取重要信号的生物特殊能力。在本技术的研究中，重点关注鸡尾酒会效应，以构建一种新型的传感系统。本文提出了用于传感器融合的三维逻辑元件阵列，即在时空坐标上集成音频信号和序列图像的三维传感器融合系统。研究总结如下。构建了一个具有16 × 16 × 8个融合单元的三维阵列的原型系统。在原型系统中使用了16万个栅极fpga。使用该系统的实验结果证明了新三维系统概念的优点。今年还开发了BPF-ICA技术，作为声源定位的初步步骤。实验证明了该算法在声源定位方面的改进，特别是在三维声源定位方面。这些结果表明了三维逻辑单元概念在时空坐标传感器融合中的优越性。

A psychological term "cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this technological research, the cocktail party effect is focused on in order to construct a new type of sensing system. Three dimensional array of logic elements for sensor fusion, i.e., three dimensional sensor fusion system which integrate audio signals and sequential images on time-space coordinates, is proposed in the research. Summaries of the research are as follows.A prototype system with a three dimensional array of 16 x 16 x 8 fusion cells has been constructed. Sixteen 60,000 gate FPGAs are used in the prototype system. Results of experiments using this system demonstrated the advantages of the concept of the new 3D system.The BPF-ICA technique was also developed in this year, which was used as a preliminary step of sound source localizations. Using this algorithm, improvements in the source localization were demonstrated experimentally, especially in the case of 3D sound source location.As the consequence of these results, the advantages of the concept of three dimensional logic elements for sensor fusion on time-space coordinates were shown.