Dynamic Interaction of Multisensory Information in 3D Object Recognition

3D 物体识别中多感官信息的动态交互

• 批准号: 17200019
• 负责人: ANDO Hiroshi
• 金额: $ 24.29万
• 依托单位: Advanced Telecommunications Research Institute International
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (A)
• 财政年份: 2005
• 资助国家: 日本
• 起止时间: 2005 至 2007
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-17200019/
• 关键词: Object recognition; Brain Mechanism; Multisensory Integration; Vision; Somatosensory Perception; Haptics; fMRI; Psychophysics

The purpose of this study is to investigate dynamic brain mechanisms of 3D object recognition with visual and somatosensory information through psychophysical and fMRI brain activity imaging experiments. In this study, we developed novel experimental methods using virtual and real 3D objects, and conducted psychophysical experiments to reveal cooperative and competitive mechanisms of visual and somatosensory interaction, and measured brain activities to find brain regions involved in 3D object learning and image generation with visual and haptic information.Regarding cooperative and competitive mechanisms of visual and somatosensory interaction, we conducted psychophysical experiments on the interpretation of ambiguous visual patterns using a visual-haptic display system and found that haptic information affects human visual perception and that binocular disparity information plays an important role in 3D visual and haptic shape estimation. We further developed multi-sensory interactio … More n system by devising new algorithms to integrate 3D image, force feedback, and contact sounds in real-time.In fMRI experiments on human object recognition, we used real objects produced by a 3D printer and measured brain activities when the subjects performed haptic object learning and top-down image generation tasks. The results suggested that the inferior parietal lobule is involved in haptic 3D object learning and BA7 and the precuneous regions are involved in haptic top-down judgment. We further conducted experiments on cross-modal top-down processing, showing that the visual object shape represented in the lateral occipital complex(LOC) could be retrieved from haptic object memory acquired in the inferior parietal lobule(BA40), and that the haptic object shape represented in the somatosensory areas could be retrieved from visual object memory acquired in the fusiform gyrus. The results also suggested that multi-sensory imagery could be represented in BA7 and the precuneous regions. Based on these findings, we developed a brain network model of human multi-sensory integration. Less

这项研究的目的是通过心理物理和fMRI脑活动成像实验研究3D对象识别的动态大脑机制。 In this study, we developed novel experimental methods using virtual and real 3D objects, and conducted psychophysical experiments to reveal coordinated and competitive mechanisms of visual and somatosensory interaction, and measured brain activities to find brain regions involved in 3D object learning and image generation with visual and hattic information.Regarding cooperative and competitive mechanisms of visual and somatosensory interaction, we conducted psychophysical experiments on the interpretation of使用视觉热显示系统的模棱两可的视觉模式，发现Hattic信息会影响人类的视觉感知，并且双眼差异信息在3D视觉和Hattic形状估计中起着重要作用。我们通过设计新算法来实时整合3D图像，强制反馈和触点声音，进一步开发了更多N系统。结果表明，下顶叶与触觉3D对象学习和BA7有关，而前后的区域参与了触觉上自上而下的法官。我们进一步进行了有关跨模式自上而下处理的实验，这表明可以从下壁叶洛布尔（BA40）中获取的枕枕形状（LOC）中所示的视觉物体形状（LOC）检索，并且可以从视觉物体内存中获得代表的触觉物体形状，可以追溯到视觉物体的形状中。结果还表明，在BA7和前后区域中可以表示多感官图像。基于这些发现，我们开发了人类多感官整合的大脑网络模型。较少的

项目成果

Human Evaluation of Visual and Haptic Interaction

视觉和触觉交互的人类评估

• 发表时间: 2007
• 期刊: Proc.Human Computer Interaction International 2007 8・LNCS4557
• 作者: Ando;Hiroshi
• 通讯作者: Hiroshi

人が感じる臨場感の知覚認識機構と評価技術

人的存在感感知识别机制及评价技术

• 发表时间: 2008
• 期刊: 電子情報通信学会2008年総合大会論文集2 BP-3-5
• 作者: 安藤;広志
• 通讯作者: 広志

Perceptual and Cognitive Mechanisms and Evaluation Methods of "Presence".

“在场”的感知认知机制及评价方法。

• 发表时间: 2008
• 期刊: Proceedings of the 2008 IEICE General Conference 2 BP-3-5
• 作者: Ando;H.
• 通讯作者: H.

Multisensory Information Processing in 3D Object Recognition.

3D 对象识别中的多感官信息处理。

• 发表时间: 2007
• 作者: Ando;H.
• 通讯作者: H.

Prism adaptation and aftereffect in darts.

飞镖中的棱镜适应和后效。

• 发表时间: 2006
• 期刊: Vision Vol.18(Supplement)
• 作者: Ashida;H. and Kitakaze;T.
• 通讯作者: T.