Role of Temporal Congruity in Visual-Haptic Cross-modal Object Recognition and Le

描述（由申请人提供）：当我们看到某人的嘴唇与他/她说话时一致时，我们会感知到这个声音是那个人发出的。长期以来，各种感官输入之间的这种时间一致性或匹配是整合来自各种感官的信息的关键因素。在许多情况下，时间一致性在产生统一的多模态感知中的作用已经得到了很好的研究，特别是在视听跨模态整合的情况下。然而，时间一致性在视觉输入与触觉（主动触摸）输入的跨模态整合中的作用仍然知之甚少。本研究的总体目标是了解时间一致性在视觉-触觉跨模态整合中的作用。为此，我们将使用视觉-触觉对象识别作为测试案例，系统地研究一致性（或缺乏一致性）对健康成人对象识别的感知影响。受试者的任务是将给定的视觉-触觉对象分类为两个给定类别之一。这些类别将被定义为精确指定的、自然分布的特征，因此，成功分类所需的底层信息处理可以使用定量的、概率的框架来定义。为了帮助创建测试该框架所需的定义良好的概率分布，我们将通过模拟生物过程来生成具有所需特征分布的新颖的“设计师”3d视觉和触觉对象类别

DESCRIPTION (provided by applicant): When we see someone's lips move consistently with his/her speech, we perceptually assign the sound to that person. It has long been clear that such temporal congruity, or match, between the various sensory inputs is a crucial factor in the integration of the information from the various senses. The role of temporal congruity in generating a unified, multimodal percept has been well studied in many cases, especially in the case of audio-visual cross-modal integration. However, the role of temporal congruity in the cross-modal integration of visual input with the haptic (active touch) input remains poorly understood. The overall goal of the proposed research is to understand the role of temporal congruity in visual-haptic cross-modal integration. To this end, we will use visual-haptic object recognition as a test case, and systematically study the perceptual effects of congruity (or lack thereof) on object recognition in healthy adult human subjects. The subjects' task will be to categorize a given visual-haptic object into one of the two given categories. The categories will be defined with precisely specified, naturalistic distribution of features, so that the underlying information processing required for successful categorization can be defined using a quantitative, probabilistic framework. To help create the well-defined probability distributions required for testing this framework, we will generate novel, 'designer' categories of 3-D visual and haptic objects with desired distributions of features by simulating the biological processes of embryogenesis and natural selection, and/or computer graphical morphing. The stimuli will be presented to the subjects so that they cannot see what they touch, and cannot touch what they see. We will wirelessly synchronize the visual image with its haptic counterpart in real time using 3-D orientation sensors. We will systematically manipulate the synchrony between the visual image and its haptic counterpart, and measure its effect on the cross-modal recognition of learned objects, as well as on cross-modal learning itself. We expect to find that lack of temporal congruity significantly affects both categorization and category learning performance. The proposed study will not only elucidate the precise role of temporal congruity in visual-haptic object recognition, but will also introduce a highly innovative and quantitative methodology with potentially important implications to basic research as well as visual-haptic rehabilitative treatments.