权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

FINDING THE CORRESPONDENCES AMONG EVENTS OBSERVED BY DIFFERENT SENSATIONS AND ITS APPLICATION TO KNOWLEDGE ACQUISITION

寻找不同感觉观察到的事件之间的对应性及其在知识获取中的应用

基本信息

批准号：
11308012
负责人：
OHNISHI Noboru
金额：
$ 13.29万
依托单位：
NAGOYA UNIVERSITY
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (A)
财政年份：
1999
资助国家：
日本
起止时间：
1999 至 2002
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-11308012/
关键词：
Sensor Fusion Visual Sense Auditory Sense Grouping Learning Motion Sound 動作触覚

项目摘要

A newborn baby can correlate mouth-movement with its speech, and acquire the ability of speaking based on this correspondence. In this study, we studied and developed novel methods for relating the events observed by different sensations such as visual and auditory senses, and the process of acquiring knowledge based on the obtained correspondence.First, we summarized the laws for grouping multi-modal events. Considering the laws, we developed a general cue for finding correspondence between visual and auditory events, i.e. cross-modal grouping, by using the similarity between motion and rhythm, and the simultaneity of onset and moving direction change. Furthermore we introduced spatio-temporal invariant for us to find correspondence even when sound emitting objects move in the environment.Next, we compared active and passive observations in knowledge acquisition. We found that the active observation is superior to the passive one because it can decrease information processing load due to selective attention and use feedback information that is the output of active operation toward the environment.We developed an active observation system. The system consists of an artificial head with a camera and a microphone, a manipulator as a hand acting to the external world and a computer. We developed and implemented software so that the system can deepen the understanding of the scene by iterating observation and motion.Finally, we conducted experiment in the real environment, where two sound emitting moving objects exist, we obtained satisfactory results showing the validity of the proposed method.

新生儿可以将嘴部运动与说话联系起来，并在这种联系的基础上获得说话的能力。在本研究中，我们研究并开发了将视觉和听觉等不同感觉所观察到的事件联系起来的新方法，以及基于所获得的对应关系获取知识的过程。首先，总结了多模态事件分组的规律。考虑到这些规律，我们开发了一种通用的线索来寻找视觉和听觉事件之间的对应关系，即跨模态分组，利用运动和节奏之间的相似性，以及开始和移动方向变化的同时性。此外，我们还引入了时空不变量，以便我们在发声物体在环境中移动时也能找到对应关系。接下来，我们比较了主动观察和被动观察在知识获取中的作用。我们发现主动观察优于被动观察，因为它可以减少由于选择性注意而产生的信息处理负荷，并利用主动操作对环境的输出反馈信息。我们开发了一个主动观测系统。该系统由一个带有摄像头和麦克风的人造头、一个作为手的操纵器和一台计算机组成。我们开发并实现了软件，使系统能够通过反复观察和运动来加深对场景的理解。最后，在两个发声运动物体存在的真实环境中进行了实验，得到了令人满意的结果，证明了所提方法的有效性。