The neural basis of multisensory processing during self motion in real and virtual environments

为了安全地与环境互动，中枢神经系统（CNS）综合我们的感官信息来指导决策和行为。由于多感觉整合影响决策和行动的执行，了解中枢神经系统如何做到这一点是理解神经功能和紊乱的基础。然而，我们对自我运动过程中感官信息的处理方式知之甚少，也不知道当多感官事件呈现冲突信息时如何预测晕动病的发生，这是目前限制虚拟现实技术应用的主要障碍。我的实验室研究人类观察者如何对时空变化中相互冲突的多感官信息做出反应。我们的工作在真实和虚拟环境中进行，使用虚拟和增强现实技术。通过使用主要的行为数据，本研究为促进我们对自我运动过程中多感觉加工的基本理解做出了重大贡献，本研究将发展三个目标：(1)自我运动过程中多感觉事件的时间加工。人类自我运动的参数评估与对自我运动的生理和神经反应相结合，将用于构建随时间推移服务于多感觉整合的神经机制的理论模型。(2)自我运动过程中的感觉线索整合。最大似然估计通常可以用来可靠地预测感官线索是如何被中枢神经系统整合的。在这里，我们将评估文献中寻找视觉和前庭线索最佳整合的差异是否源于自我运动期间视觉敏感性的变化和感觉线索之间的感知时间延迟，以及与感知决策相关的神经活动记录中的个体差异的作用。(3)自我运动时的晕动病对策。我们的工作表明，除了自我运动阈值等其他因素外，主要通过评估平衡控制的视光流来预测晕动病易感性的潜力。其他研究也显示了对晕动病的类似预测，利用生理学记录对感觉冲突的反应，如心率、出汗、呼吸和胃反应。在这里，我们建议结合这些方法来评估是否可以更好地预测真实和虚拟环境中的晕动病。HQP将获得分析、技术和演讲技能，在当今的技能和知识经济中具有很强的竞争力。了解多感官信息是如何被处理的，以及它是如何影响感知决策和行动的，这不仅对理解中枢神经系统是如何工作的至关重要，而且对于减少晕动病的发生率，以及在感觉冲突环境中产生的错误，这些错误阻碍了航空、航天、设计、移动计算和医疗保健行业更广泛地采用节省成本的技术。

To safely interact with the environment, the central nervous system (CNS) synthesizes information across our senses to guide decision-making and behaviour. As multisensory integration affects how decisions and actions are executed, knowing how the CNS does this is fundamental to understanding neural function and disorder. However, we know relatively little about how sensory information is processed during self-motion or how to predict motion sickness incidence when multisensory events present conflicting information, which is currently the main limiting barrier of virtual reality technology adoption. My lab researches how human observers respond to conflicting multisensory information that varies in space and time. Our work is conducted in both real and in virtual environments using virtual and augmented reality technology. Having made significant contributions to advancing our basic understanding of multisensory processing during self-motion using primarily behavioural data, this proposal will develop three objectives: (1) Temporal processing of multisensory events during self-motion. Parametric assessment of human self-motion paired with physiological and neural responses to self-motion will be used to construct a theoretical model of the neural mechanisms that sub serve multisensory integration over time. (2) Sensory cue integration during self-motion. Maximum-likelihood estimation can often be used to reliably predict how sensory cues are integrated by the CNS. Here, we will assess whether discrepancies in finding optimal integration of visual and vestibular cues in the literature result from change in visual sensitivity during self-motion and perceived temporal delays between sensory cues, as well as the role of individual differences in the neural activity recorded that is associated with perceptual decision-making. (3) Motion sickness countermeasures during self-motion. Our work has demonstrated the potential of predicting motion sickness susceptibility primarily from assessments of visual optic flow on balance control in addition to other factors such as thresholds for self-motion. Other work has shown comparable predictions for motion sickness, using physiological recordings in response to sensory conflict such as heart rate, perspiration, respiration, and gastric responses. Here we propose to combine these methods to assess whether motion sickness in real and virtual environments can be better predicted. HQP will gain analytical, technical, and presentation skills to be highly competitive in today's skills and knowledge-based economy. Knowing how multisensory information is processed and how it affects perceptual decision-making and action is critical not only to understand how the CNS works but also to reduce the incidence of motion sickness, and errors made in sensory conflicting environments which is holding back wider adoption of cost-saving technology in the aviation, space, design, mobile computing, and healthcare industries.