Cocktail party effect procesor using audio-visual sensor fusion technique

采用视听传感器融合技术的鸡尾酒会效果处理器

基本信息

批准号：
08455194
负责人：
TAKAHASHI Kota
金额：
$ 5.25万
依托单位：
The University of Electro-Communications
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (B)
财政年份：
1996
资助国家：
日本
起止时间：
1996 至 1997
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-08455194/
关键词：
image processing parallel processing FPGA

项目摘要

The term "Cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this research, the cocktail party effect is focused on in order to construct a new type of sensing system. Two dimensional array of logic elements for sensor fusion, i.e., two dimensional sensor fusion system which integrate audio signals and sequential images on time-space coordinates, is proposed in this research. Summaries of the research are as followas.A small prototype system wiht an array of 8x8, which had been made as preliminary system, was extended to a larger system with an array of 24x16 inthis year. Results of experiments using the revised system demonstraed the advantages of the concept of the new system, especially in the case of multiplexed data.Automatic pitch tracking algorithm was also developed in this year, which was used as a preliminary step of sound source localization. Using this algorithm, improvements in the source localization subsystem were demonstrated experimentally.The other subsystem is an image processing part. In this part, a new algorithm using feature points was developed, which wa based on the theory of independent component analysis. Using this type of feature points, improvements in the stereo vision subsystem were demonstrated experimentally.As the consequence of these results, the advantages of the concept of two dimensional logic elements for sensor fusion on time-space coordinates were shown. In the future, the system will be revised in order to evaluate the efficiency of the algorithm more quantitatively.

“鸡尾酒会效应”一词是指从其他环境噪音中提取重要信号的生物学特殊能力。在这项研究中，鸡尾酒会的效果旨在构建一种新型的传感系统。在这项研究中，提出了用于传感器融合的逻辑元素的逻辑元素，即在时间空间坐标上整合音频信号和顺序图像的二维传感器融合系统。该研究的摘要如下。一个小型原型系统，没有8x8的阵列，该系统已作为初步系统制成，扩展到了一个较大的系统，在这一年中，阵列为24x16。使用修订系统的实验结果证明了新系统概念的优势，尤其是在多重数据的情况下。自动音调跟踪算法也在今年开发了，该算法被用作声源定位的初步步骤。使用该算法，通过实验证明了源定位子系统的改进。其他子系统是图像处理部分。在这一部分中，开发了一种新的使用特征点的算法，该算法基于独立组件分析理论。使用这种特征点，通过实验证明了立体声视觉子系统的改进。由于这些结果，显示了在时空坐标上传感器融合的两个维逻辑元素概念的优势。将来，将对系统进行修订，以便更定量地评估算法的效率。