Three dimensional cocktail party effect processor using audio-visual sensor fusion technique

心理学术语“鸡尾酒会效应”指的是从其他环境噪音中提取重要信号的生物特殊能力。在这项技术研究中，重点研究了鸡尾酒会效应，以期构建一种新型的传感系统。提出了一种用于传感器融合的三维逻辑单元阵列，即在时空坐标上集成音频信号和序列图像的三维传感器融合系统。主要研究内容如下：构建了一个16×16×8融合细胞三维阵列的原型系统。在原型系统中使用了16个6万门的现场可编程门阵列。使用该系统的实验结果证明了新的3D系统概念的优势。今年还开发了BPF-ICA技术，作为声源定位的初步步骤。通过实验验证了该算法在声源定位方面的改进，特别是在三维声源定位的情况下，展示了三维逻辑单元概念在时空坐标下进行传感器融合的优势。

A psychological term "cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this technological research, the cocktail party effect is focused on in order to construct a new type of sensing system. Three dimensional array of logic elements for sensor fusion, i.e., three dimensional sensor fusion system which integrate audio signals and sequential images on time-space coordinates, is proposed in the research. Summaries of the research are as follows.A prototype system with a three dimensional array of 16 x 16 x 8 fusion cells has been constructed. Sixteen 60,000 gate FPGAs are used in the prototype system. Results of experiments using this system demonstrated the advantages of the concept of the new 3D system.The BPF-ICA technique was also developed in this year, which was used as a preliminary step of sound source localizations. Using this algorithm, improvements in the source localization were demonstrated experimentally, especially in the case of 3D sound source location.As the consequence of these results, the advantages of the concept of three dimensional logic elements for sensor fusion on time-space coordinates were shown.