Neural Basis of Multisensory Integration

神经系统可以通过各种感觉模态来感知世界，这些感觉模态提供关于真实的世界的各个方面的补充信息以及冗余信息。通过一个被称为多感觉整合的过程，这些感觉信号被合并并用于执行各种任务，例如定位空间中的物体或识别说话者发出的单词。在美国国家科学基金会的资助下，Alexandre Pouget博士正在进行一项强有力的理论驱动的研究计划，结合建模，人类心理物理学和非人类灵长类动物神经生理学研究，以调查被称为多感觉整合的过程的神经基础。两个因素使得多感觉整合困难：第一，感觉形式通常是不同的格式（例如，同一物体的声音和图像不能直接比较）;以及第二，感觉形态不是同样可靠的（例如，通常基于声音比基于嘴唇运动更容易辨别说话者说出了哪个词）。使用贝叶斯推理的框架，研究计划是首先开发一个多感觉整合的神经理论，可以最佳地解决这两个问题。然后，计划是在清醒的猴子的多感觉区域进行记录，以测试该方法的有效性。虽然直觉认为神经元应该对空间中的相同位置做出反应，而不管模态如何，但初步模型却表明并非如此。在计算网络中，给定神经元的视觉和触觉感受野并不占据完全相同的位置。实验将涉及皮层视觉触觉神经元的记录，以测试它们的感受野是否像模拟预测的那样表现。 该理论的时间方面也将通过人类心理物理学实验进行测试。将要求受试者在存在人工运动错误的情况下进行眼球运动序列。最佳整合理论预测，受试者将试图纠正他们的错误成比例的视觉反馈的可靠性。这项研究将涉及博士后，本科生和研究生，其结果将在主要的多学科会议。研究结果的意义也将远远超出多感官整合。事实上，多感觉整合是贝叶斯推理一般问题的一个子案例，贝叶斯推理被认为是许多认知过程的核心，如物体识别，视觉感知，运动控制和抽象推理。

The nervous system can sense the world through a variety of sensory modalities providing complementary as well as redundant information about various aspects of the real world. Through a process known as multisensory integration, those sensory signals are merged and used to perform a variety of tasks such as locating objects in space or identifying the words uttered by a speaker. With funding from the National Science Foundation, Dr. Alexandre Pouget is conducting a strongly theory-driven research program combining modeling, human psychophysics and non-human primate neurophysiologal studies to investigate the neural basis of the process known as multisensory integration. Two factors make multisensory integration difficult: First, the sensory modalities are often in different formats (e.g., the sound and image of the same object are not directly comparable); and Second, the sensory modalities are not equally reliable (e.g., it is typically much easier to tell which word is uttered by a speaker based on sound than on lip movements). Using the framework of Bayesian inference, the research plan is to first develop a neural theory of multisensory integration that can solve both problems optimally. Then the plan is to perform recording in multisensory areas of awake monkeys to test the validity of the approach. While intuition would suggest that a neuron should respond to the same location in space regardless of the modality, a preliminary model suggests otherwise. In the computational network, the visual and tactile receptive fields of a given neuron do not occupy the exact same location. Experiments will involve recordings from cortical visuo-tactile neurons to test whether their receptive fields behave as predicted by simulations. The temporal aspect of the theory will also be tested through psychophysics experiments in humans. Subjects will be asked to perform sequences of eye movements in the presence of artificial motor error. The theory of optimal integration predicts that subjects will attempt to correct for their errors in proportion to the reliability of the visual feedback.This research will involve postdocs, undergraduate and graduate students, and the results will be presented at major multidisciplinary conferences. The results will also have implications well beyond multisensory integration. Indeed, multisensory integration is a subcase of the general problem of Bayesian inference, which is believed to be at the heart of numerous cognitive processes such as object recognition, visual perception, motor control and abstract reasoning.