Role of Temporal Congruity in Visual-Haptic Cross-modal Object Recognition and Le

描述(由申请者提供)：当我们看到某人的嘴唇随着他/她的讲话而一致地移动时，我们会感知地将声音赋予此人。很明显，不同感觉输入之间的这种时间一致性或匹配是整合来自不同感觉的信息的关键因素。时间一致性在产生统一的多通道知觉中的作用已经在许多情况下得到了很好的研究，特别是在视听跨通道整合的情况下。然而，时间一致性在视觉输入和触觉(主动触摸)输入的跨模式整合中的作用仍然知之甚少。这项研究的总体目标是了解时间一致性在视觉-触觉跨通道整合中的作用。为此，我们将以视觉-触觉物体识别为测试案例，系统地研究一致性(或缺乏一致性)对健康成人受试者物体识别的知觉效应。受试者的任务将是将给定的视觉触觉对象归入两个给定类别中的一个。这些类别将用精确指定的、自然的特征分布来定义，以便可以使用量化的、概率的框架来定义成功分类所需的基本信息处理。为了帮助创建测试该框架所需的明确定义的概率分布，我们将通过模拟生物过程来生成具有所需特征分布的新的3D视觉和触觉对象类别 胚胎发生和自然选择，和/或计算机图形变形。刺激将被呈现给受试者，这样他们就看不到他们触摸的东西，也不能触摸他们看到的东西。我们将使用无线方式将视觉图像与其触觉图像实时同步 3-D方位传感器。我们将系统地操纵视觉图像和触觉图像之间的同步性，并测量其对学习对象的跨模式识别以及跨模式学习本身的影响。我们期望发现，缺乏时间一致性会显著影响分类和类别学习成绩。这项拟议的研究不仅将阐明时间一致性在视觉-触觉对象识别中的确切作用，而且还将引入一种高度创新的定量方法，对基础研究和视觉-触觉康复治疗具有潜在的重要意义。

DESCRIPTION (provided by applicant): When we see someone's lips move consistently with his/her speech, we perceptually assign the sound to that person. It has long been clear that such temporal congruity, or match, between the various sensory inputs is a crucial factor in the integration of the information from the various senses. The role of temporal congruity in generating a unified, multimodal percept has been well studied in many cases, especially in the case of audio-visual cross-modal integration. However, the role of temporal congruity in the cross-modal integration of visual input with the haptic (active touch) input remains poorly understood. The overall goal of the proposed research is to understand the role of temporal congruity in visual-haptic cross-modal integration. To this end, we will use visual-haptic object recognition as a test case, and systematically study the perceptual effects of congruity (or lack thereof) on object recognition in healthy adult human subjects. The subjects' task will be to categorize a given visual-haptic object into one of the two given categories. The categories will be defined with precisely specified, naturalistic distribution of features, so that the underlying information processing required for successful categorization can be defined using a quantitative, probabilistic framework. To help create the well-defined probability distributions required for testing this framework, we will generate novel, 'designer' categories of 3-D visual and haptic objects with desired distributions of features by simulating the biological processes of embryogenesis and natural selection, and/or computer graphical morphing. The stimuli will be presented to the subjects so that they cannot see what they touch, and cannot touch what they see. We will wirelessly synchronize the visual image with its haptic counterpart in real time using 3-D orientation sensors. We will systematically manipulate the synchrony between the visual image and its haptic counterpart, and measure its effect on the cross-modal recognition of learned objects, as well as on cross-modal learning itself. We expect to find that lack of temporal congruity significantly affects both categorization and category learning performance. The proposed study will not only elucidate the precise role of temporal congruity in visual-haptic object recognition, but will also introduce a highly innovative and quantitative methodology with potentially important implications to basic research as well as visual-haptic rehabilitative treatments.