Dynamic Interaction of Multisensory Information in 3D Object Recognition

3D 物体识别中多感官信息的动态交互

• 批准号: 17200019
• 负责人: ANDO Hiroshi
• 金额: $ 24.29万
• 依托单位: Advanced Telecommunications Research Institute International
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (A)
• 财政年份: 2005
• 资助国家: 日本
• 起止时间: 2005 至 2007
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-17200019/
• 关键词: Object recognition; Brain Mechanism; Multisensory Integration; Vision; Somatosensory Perception; Haptics; fMRI; Psychophysics

The purpose of this study is to investigate dynamic brain mechanisms of 3D object recognition with visual and somatosensory information through psychophysical and fMRI brain activity imaging experiments. In this study, we developed novel experimental methods using virtual and real 3D objects, and conducted psychophysical experiments to reveal cooperative and competitive mechanisms of visual and somatosensory interaction, and measured brain activities to find brain regions involved in 3D object learning and image generation with visual and haptic information.Regarding cooperative and competitive mechanisms of visual and somatosensory interaction, we conducted psychophysical experiments on the interpretation of ambiguous visual patterns using a visual-haptic display system and found that haptic information affects human visual perception and that binocular disparity information plays an important role in 3D visual and haptic shape estimation. We further developed multi-sensory interactio … More n system by devising new algorithms to integrate 3D image, force feedback, and contact sounds in real-time.In fMRI experiments on human object recognition, we used real objects produced by a 3D printer and measured brain activities when the subjects performed haptic object learning and top-down image generation tasks. The results suggested that the inferior parietal lobule is involved in haptic 3D object learning and BA7 and the precuneous regions are involved in haptic top-down judgment. We further conducted experiments on cross-modal top-down processing, showing that the visual object shape represented in the lateral occipital complex(LOC) could be retrieved from haptic object memory acquired in the inferior parietal lobule(BA40), and that the haptic object shape represented in the somatosensory areas could be retrieved from visual object memory acquired in the fusiform gyrus. The results also suggested that multi-sensory imagery could be represented in BA7 and the precuneous regions. Based on these findings, we developed a brain network model of human multi-sensory integration. Less

本研究的目的是通过心理物理和功能磁共振脑活动成像实验，探讨视觉和体感信息下三维物体识别的脑动力学机制。在本研究中，我们开发了使用虚拟和真实的3D物体的新实验方法，并进行了心理物理学实验，以揭示视觉和体感互动的合作和竞争机制，并测量大脑活动，以发现参与3D物体学习和图像生成的大脑区域。关于视觉和体感互动的合作和竞争机制，我们使用视觉-触觉显示系统对模糊视觉模式的解释进行了心理物理学实验，发现触觉信息影响人类的视觉感知，并且双眼视差信息在3D视觉和触觉形状估计中起着重要作用。我们进一步开发了多感官互动， ...更多信息 在人类物体识别的fMRI实验中，我们使用3D打印机产生的真实的物体，并测量受试者执行触觉物体学习和自上而下的图像生成任务时的大脑活动。结果表明，顶叶下小叶参与触觉三维物体学习，BA 7和前意识区参与触觉自上而下判断。我们进一步进行了跨通道自上而下的加工实验，结果表明，在枕外侧复合体中表示的视觉对象形状可以从顶下小叶中获得的触觉对象记忆中提取，而在躯体感觉区中表示的触觉对象形状可以从梭状回中获得的视觉对象记忆中提取。结果还表明，多感觉表象在BA 7和前意识区都有表现。基于这些发现，我们开发了一个人类多感觉整合的脑网络模型。少

项目成果

Human Evaluation of Visual and Haptic Interaction

视觉和触觉交互的人类评估

• 发表时间: 2007
• 期刊: Proc.Human Computer Interaction International 2007 8・LNCS4557
• 作者: Ando;Hiroshi
• 通讯作者: Hiroshi

Prism adaptation and aftereffect in darts.

飞镖中的棱镜适应和后效。

• 发表时间: 2006
• 期刊: Vision Vol.18(Supplement)
• 作者: Ashida;H. and Kitakaze;T.
• 通讯作者: T.

Multisensory Information Processing in 3D Object Recognition.

3D 对象识别中的多感官信息处理。

• 发表时间: 2007
• 作者: Ando;H.
• 通讯作者: H.

人が感じる臨場感の知覚認識機構と評価技術

人的存在感感知识别机制及评价技术

• 发表时间: 2008
• 期刊: 電子情報通信学会2008年総合大会論文集2 BP-3-5
• 作者: 安藤;広志
• 通讯作者: 広志

Perceptual and Cognitive Mechanisms and Evaluation Methods of "Presence".

“在场”的感知认知机制及评价方法。

• 发表时间: 2008
• 期刊: Proceedings of the 2008 IEICE General Conference 2 BP-3-5
• 作者: Ando;H.
• 通讯作者: H.