Cocktail party effect procesor using audio-visual sensor fusion technique

点击翻译按钮获取中文摘要

The term "Cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this research, the cocktail party effect is focused on in order to construct a new type of sensing system. Two dimensional array of logic elements for sensor fusion, i.e., two dimensional sensor fusion system which integrate audio signals and sequential images on time-space coordinates, is proposed in this research. Summaries of the research are as followas.A small prototype system wiht an array of 8x8, which had been made as preliminary system, was extended to a larger system with an array of 24x16 inthis year. Results of experiments using the revised system demonstraed the advantages of the concept of the new system, especially in the case of multiplexed data.Automatic pitch tracking algorithm was also developed in this year, which was used as a preliminary step of sound source localization. Using this algorithm, improvements in the source localization subsystem were demonstrated experimentally.The other subsystem is an image processing part. In this part, a new algorithm using feature points was developed, which wa based on the theory of independent component analysis. Using this type of feature points, improvements in the stereo vision subsystem were demonstrated experimentally.As the consequence of these results, the advantages of the concept of two dimensional logic elements for sensor fusion on time-space coordinates were shown. In the future, the system will be revised in order to evaluate the efficiency of the algorithm more quantitatively.