Dynamic Interaction of Multisensory Information in 3D Object Recognition

3D 物体识别中多感官信息的动态交互

• 批准号: 17200019
• 负责人: ANDO Hiroshi
• 金额: $ 24.29万
• 依托单位: Advanced Telecommunications Research Institute International
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (A)
• 财政年份: 2005
• 资助国家: 日本
• 起止时间: 2005 至 2007
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-17200019/
• 关键词: Object recognition; Brain Mechanism; Multisensory Integration; Vision; Somatosensory Perception; Haptics; fMRI; Psychophysics

The purpose of this study is to investigate dynamic brain mechanisms of 3D object recognition with visual and somatosensory information through psychophysical and fMRI brain activity imaging experiments. In this study, we developed novel experimental methods using virtual and real 3D objects, and conducted psychophysical experiments to reveal cooperative and competitive mechanisms of visual and somatosensory interaction, and measured brain activities to find brain regions involved in 3D object learning and image generation with visual and haptic information.Regarding cooperative and competitive mechanisms of visual and somatosensory interaction, we conducted psychophysical experiments on the interpretation of ambiguous visual patterns using a visual-haptic display system and found that haptic information affects human visual perception and that binocular disparity information plays an important role in 3D visual and haptic shape estimation. We further developed multi-sensory interactio … More n system by devising new algorithms to integrate 3D image, force feedback, and contact sounds in real-time.In fMRI experiments on human object recognition, we used real objects produced by a 3D printer and measured brain activities when the subjects performed haptic object learning and top-down image generation tasks. The results suggested that the inferior parietal lobule is involved in haptic 3D object learning and BA7 and the precuneous regions are involved in haptic top-down judgment. We further conducted experiments on cross-modal top-down processing, showing that the visual object shape represented in the lateral occipital complex(LOC) could be retrieved from haptic object memory acquired in the inferior parietal lobule(BA40), and that the haptic object shape represented in the somatosensory areas could be retrieved from visual object memory acquired in the fusiform gyrus. The results also suggested that multi-sensory imagery could be represented in BA7 and the precuneous regions. Based on these findings, we developed a brain network model of human multi-sensory integration. Less

本研究的目的是通过心理物理和fMRI脑活动成像实验，研究视觉和体感信息下3D物体识别的动态脑机制。在本研究中，我们开发了新颖的使用虚拟和真实3D对象的实验方法，并进行了心理物理实验，以揭示视觉和体感交互的合作和竞争机制，并测量了大脑活动，以发现参与3D对象学习和图像生成的大脑区域和视觉和触觉信息。关于视觉和体感交互的合作和竞争机制，我们使用视觉-触觉显示系统进行了模糊视觉模式的心理物理实验，发现触觉信息影响人类视觉感知，双目视差信息在3D视觉和触觉形状估计中起着重要作用。我们进一步开发了多感觉交互…在人体物体识别的fMRI实验中，我们使用了3D打印机产生的真实物体，并测量了受试者在执行触觉物体学习和自上而下图像生成任务时的大脑活动。结果提示，顶下小叶参与了触觉3D物体的学习，Ba7和前额叶参与了触觉自上而下的判断。我们进一步进行了跨模式自上而下加工的实验，结果表明，枕外侧复合体(LOC)的视觉物体形状可以从顶下小叶(BA40)获得的触觉物体记忆中提取出来，而体感区域的触觉物体形状可以从梭形回获得的视觉物体记忆中提取出来。结果还表明，多感觉表象可能存在于Ba7区和前额叶区域。基于这些发现，我们开发了一个人类多感觉整合的大脑网络模型。较少

项目成果

Human Evaluation of Visual and Haptic Interaction

视觉和触觉交互的人类评估

• 发表时间: 2007
• 期刊: Proc.Human Computer Interaction International 2007 8・LNCS4557
• 作者: Ando;Hiroshi
• 通讯作者: Hiroshi

Prism adaptation and aftereffect in darts.

飞镖中的棱镜适应和后效。

• 发表时间: 2006
• 期刊: Vision Vol.18(Supplement)
• 作者: Ashida;H. and Kitakaze;T.
• 通讯作者: T.

Multisensory Information Processing in 3D Object Recognition.

3D 对象识别中的多感官信息处理。

• 发表时间: 2007
• 作者: Ando;H.
• 通讯作者: H.

人が感じる臨場感の知覚認識機構と評価技術

人的存在感感知识别机制及评价技术

• 发表时间: 2008
• 期刊: 電子情報通信学会2008年総合大会論文集2 BP-3-5
• 作者: 安藤;広志
• 通讯作者: 広志

Perceptual and Cognitive Mechanisms and Evaluation Methods of "Presence".

“在场”的感知认知机制及评价方法。

• 发表时间: 2008
• 期刊: Proceedings of the 2008 IEICE General Conference 2 BP-3-5
• 作者: Ando;H.
• 通讯作者: H.