Dynamic Interaction of Multisensory Information in 3D Object Recognition

3D 物体识别中多感官信息的动态交互

• 批准号: 17200019
• 负责人: ANDO Hiroshi
• 金额: $ 24.29万
• 依托单位: Advanced Telecommunications Research Institute International
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (A)
• 财政年份: 2005
• 资助国家: 日本
• 起止时间: 2005 至 2007
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-17200019/
• 关键词: Object recognition; Brain Mechanism; Multisensory Integration; Vision; Somatosensory Perception; Haptics; fMRI; Psychophysics

The purpose of this study is to investigate dynamic brain mechanisms of 3D object recognition with visual and somatosensory information through psychophysical and fMRI brain activity imaging experiments. In this study, we developed novel experimental methods using virtual and real 3D objects, and conducted psychophysical experiments to reveal cooperative and competitive mechanisms of visual and somatosensory interaction, and measured brain activities to find brain regions involved in 3D object learning and image generation with visual and haptic information.Regarding cooperative and competitive mechanisms of visual and somatosensory interaction, we conducted psychophysical experiments on the interpretation of ambiguous visual patterns using a visual-haptic display system and found that haptic information affects human visual perception and that binocular disparity information plays an important role in 3D visual and haptic shape estimation. We further developed multi-sensory interactio … More n system by devising new algorithms to integrate 3D image, force feedback, and contact sounds in real-time.In fMRI experiments on human object recognition, we used real objects produced by a 3D printer and measured brain activities when the subjects performed haptic object learning and top-down image generation tasks. The results suggested that the inferior parietal lobule is involved in haptic 3D object learning and BA7 and the precuneous regions are involved in haptic top-down judgment. We further conducted experiments on cross-modal top-down processing, showing that the visual object shape represented in the lateral occipital complex(LOC) could be retrieved from haptic object memory acquired in the inferior parietal lobule(BA40), and that the haptic object shape represented in the somatosensory areas could be retrieved from visual object memory acquired in the fusiform gyrus. The results also suggested that multi-sensory imagery could be represented in BA7 and the precuneous regions. Based on these findings, we developed a brain network model of human multi-sensory integration. Less

本研究的目的是通过心理物理学和功能磁共振成像大脑活动成像实验，研究利用视觉和体感信息识别 3D 物体的动态大脑机制。在这项研究中，我们开发了使用虚拟和真实3D物体的新颖实验方法，并进行心理物理实验来揭示视觉和体感交互的合作和竞争机制，并测量大脑活动以找到参与3D物体学习和视觉和触觉信息图像生成的大脑区域。关于视觉和体感交互的合作和竞争机制，我们进行了解释视觉和体感交互的合作和竞争机制的心理物理实验。 使用视觉-触觉显示系统研究模糊的视觉模式，发现触觉信息影响人类视觉感知，并且双眼视差信息在 3D 视觉和触觉形状估计中发挥重要作用。我们通过设计新的算法来实时集成 3D 图像、力反馈和接触声音，进一步开发了多感官交互系统。在人体物体识别的 fMRI 实验中，我们使用 3D 打印机生产的真实物体，并测量受试者执行触觉物体学习和自上而下的图像生成任务时的大脑活动。结果表明，顶下小叶参与触觉 3D 物体学习，BA7 和楔前区参与触觉自上而下的判断。我们进一步进行了跨模态自上而下处理的实验，结果表明，枕外侧复合体（LOC）中表示的视觉对象形状可以从顶下小叶（BA40）获得的触觉对象记忆中检索出来，体感区域中表示的触觉对象形状可以从梭状回中获得的视觉对象记忆中检索出来。结果还表明，BA7 和楔前区可以表现出多感官意象。基于这些发现，我们开发了人类多感官整合的大脑网络模型。较少的

项目成果

Human Evaluation of Visual and Haptic Interaction

视觉和触觉交互的人类评估

• 发表时间: 2007
• 期刊: Proc.Human Computer Interaction International 2007 8・LNCS4557
• 作者: Ando;Hiroshi
• 通讯作者: Hiroshi

Prism adaptation and aftereffect in darts.

飞镖中的棱镜适应和后效。

• 发表时间: 2006
• 期刊: Vision Vol.18(Supplement)
• 作者: Ashida;H. and Kitakaze;T.
• 通讯作者: T.

Multisensory Information Processing in 3D Object Recognition.

3D 对象识别中的多感官信息处理。

• 发表时间: 2007
• 作者: Ando;H.
• 通讯作者: H.

人が感じる臨場感の知覚認識機構と評価技術

人的存在感感知识别机制及评价技术

• 发表时间: 2008
• 期刊: 電子情報通信学会2008年総合大会論文集2 BP-3-5
• 作者: 安藤;広志
• 通讯作者: 広志

Perceptual and Cognitive Mechanisms and Evaluation Methods of "Presence".

“在场”的感知认知机制及评价方法。

• 发表时间: 2008
• 期刊: Proceedings of the 2008 IEICE General Conference 2 BP-3-5
• 作者: Ando;H.
• 通讯作者: H.