Role of Temporal Congruity in Visual-Haptic Cross-modal Object Recognition and Le

描述（由申请人提供）：当我们看到某人的嘴唇移动与他/她的讲话一致时，我们将声音分配给那个人。很久以前就已经很清楚，各种感觉输入之间的这种时间一致性或匹配是整合各种感觉信息的关键因素。在许多情况下，时间一致性在产生统一的多模态感知中的作用已经得到了很好的研究，特别是在视听跨模态整合的情况下。然而，在视觉输入与触觉（主动触摸）输入的跨模态整合的时间一致性的作用仍然知之甚少。该研究的总体目标是了解时间一致性在视觉-触觉跨模态整合中的作用。为此，我们将使用视觉-触觉对象识别作为测试案例，并系统地研究一致性（或缺乏一致性）对健康成年人对象识别的感知影响。受试者的任务是将给定的视觉-触觉对象归类为两个给定类别之一。类别将被定义为精确指定的，自然分布的功能，以便成功分类所需的基本信息处理可以使用定量，概率框架来定义。为了帮助创建测试这个框架所需的定义良好的概率分布，我们将通过模拟生物过程， 胚胎发生和自然选择，和/或计算机图形变形。刺激将被呈现给受试者，这样他们就看不到他们触摸到的东西，也不能触摸到他们看到的东西。我们将无线同步视觉图像与其触觉对应物在真实的时间使用 三维方位传感器我们将系统地操纵视觉图像和触觉对应物之间的同步，并测量其对学习对象的跨模态识别以及跨模态学习本身的影响。我们期望发现，缺乏时间一致性显着影响分类和类别学习成绩。这项研究不仅将阐明时间一致性在视觉触觉物体识别中的确切作用，而且还将引入一种高度创新和定量的方法，对基础研究以及视觉触觉康复治疗具有潜在的重要意义。

DESCRIPTION (provided by applicant): When we see someone's lips move consistently with his/her speech, we perceptually assign the sound to that person. It has long been clear that such temporal congruity, or match, between the various sensory inputs is a crucial factor in the integration of the information from the various senses. The role of temporal congruity in generating a unified, multimodal percept has been well studied in many cases, especially in the case of audio-visual cross-modal integration. However, the role of temporal congruity in the cross-modal integration of visual input with the haptic (active touch) input remains poorly understood. The overall goal of the proposed research is to understand the role of temporal congruity in visual-haptic cross-modal integration. To this end, we will use visual-haptic object recognition as a test case, and systematically study the perceptual effects of congruity (or lack thereof) on object recognition in healthy adult human subjects. The subjects' task will be to categorize a given visual-haptic object into one of the two given categories. The categories will be defined with precisely specified, naturalistic distribution of features, so that the underlying information processing required for successful categorization can be defined using a quantitative, probabilistic framework. To help create the well-defined probability distributions required for testing this framework, we will generate novel, 'designer' categories of 3-D visual and haptic objects with desired distributions of features by simulating the biological processes of embryogenesis and natural selection, and/or computer graphical morphing. The stimuli will be presented to the subjects so that they cannot see what they touch, and cannot touch what they see. We will wirelessly synchronize the visual image with its haptic counterpart in real time using 3-D orientation sensors. We will systematically manipulate the synchrony between the visual image and its haptic counterpart, and measure its effect on the cross-modal recognition of learned objects, as well as on cross-modal learning itself. We expect to find that lack of temporal congruity significantly affects both categorization and category learning performance. The proposed study will not only elucidate the precise role of temporal congruity in visual-haptic object recognition, but will also introduce a highly innovative and quantitative methodology with potentially important implications to basic research as well as visual-haptic rehabilitative treatments.