The computation of multimodal representation through dynamic interaction

研究了手部运动和视觉反馈之间的相互作用。结果表明，中枢神经系统根据对随后的运动准确性的预测或估计来控制运动，同时考虑了感觉和运动误差。补充前运动区(Pre-SMA)、右侧顶叶后皮质(PPC)和右侧颞顶交界区(TPJ)分别参与内部运动预测、视觉运动误差在线评估和运动估计以补偿视觉反馈延迟。视觉和触觉与动态物体之间的跨通道整合实验表明，将动力学表征为感官线索的加权线性总和可以调节跨通道整合，并且整合中使用的时间线索与同时性判断中使用的时间线索无关。跨通道目标识别实验表明，在单通道…中识别3-D目标更多的是利用视点依赖和视点不变的信息，而非学习中使用的通道的物体识别只利用视点不变的信息。研究发现，熟悉物体的运动程序在以常规方式执行动作时被高度激活，而对于不熟悉的动作，视觉反馈的作用更大。这些结果表明，熟悉物体的运动表征是基于先前要抓住的物体部分与特定手形之间的相互作用。实验表明，使用轨迹球主动探索对象视图有助于对象识别。这表明，物体的视觉转换规则是通过运动系统的手部运动提供的信息和视觉信息之间的相互作用来学习的。此外，研究还表明，在估计左侧顶叶皮质电流偶极子的时间段，积极的探测会导致左侧大脑半球磁场的幅度显著降低。较少

Interactions between hand movements and visual feedback are investigated. Results suggest that the central nervous system controls movements based on the prediction or the estimation of consequent motor accuracy taking into account both sensory and motor error. It is also suggested that the pre-supplementary motor area (pre-SMA), the right posterior parietal cortex (PPC), and the right temporo-parietal junction (TPJ) are involved in internal movement prediction, online evaluation of visuomotor error, and movement estimation to compensate for visual feedback delay, respectively. Experiments on cross-modal integration between vision and haptics with dynamic objects revealed that representation of dynamics as a weighted linear summation of sensory cues mediates cross-modal integration, and that temporal cues used in integration is independent of those used in simultaneity judgments. Experiments on cross-modal object recognition showed that recognition of 3-D objects within single modality … More utilized both viewpoint-dependent and viewpoint-invariant information, whereas object recognition by a modality not used in learning exclusively utilized viewpoint-invariant information.It was found that the motor program for a familiar object is highly activated during the execution of the action in conventional manner, while the role of visual feedback is more dominant for an unfamiliar action. These results suggest that the motor representation for a familiar object is based on the previous interactions between the part of the object to be grasped and a particular hand shape. It was experimentally shown that active exploration of object views using a track ball facilitates object recognition. It suggested that a rule for view transformation of objects was learned by the interaction between information provided by hand movements from the motor system and visual information. Moreover, it was demonstrated that active exploration leads to a significant decrease of the amplitude of the magnetic field in the left hemisphere at the time period when the current dipole was estimated in the left parietal cortex. Less