Psychological experiment for clarifying the neural mechanism of complementary interaction between visual and haptic information in their binding process

点击翻译按钮获取中文摘要

When a person looks at an object while exploring it with their hand, vision frequently dominates the integrated visual-haptic perception, but in some circumstances the process is clearly affected by haptics. The aim of this study is to investigate the neural mechanism of the complementary interaction between visual and haptic information in their binding process, based on psychological experiment.1.Psychological experimentsWe conducted psychological experiments in which virtual spaces were constructed with a haptic device and 3D-visual system. The task of the psychological experiment is as follow : The subject feels the weight of a cube presented on a computer screen by manipulating the haptic device, and then feels the weight of an another cube in the same way, which has different visual characters (i.e.size, color) from the cube firstly presented. The subject has to make a decision of which cube has larger weight. We examined the effect of the visual characters on the weight-discrimination task. We showed that the visual characters of cube cause significant changes in the discrimination task, supporting a hypothesis that the visual information is rapidly processed and then act as a prediction signal in binding process of visual and haptic information.2.Neural network model for binding visual and haptic informationTo investigate the neural mechanism of sensory information processing, we made neural models for visual, auditory, and electrosensory systems. Based on the modeling study, we made a computational model for binding process of visual and haptic information. The network model of visual system was made based on backpropagation network, and the output of the network affects the network of haptic system as a prediction signal. The computational model reproduced qualitatively our psychological data described in section 1.