Multi-sensory integration across reference frames

点击翻译按钮获取中文摘要

Multi-sensory integration is the process by which multiple redundant or partially redundant pieces of sensory information are combined in order to provide a better estimate of a measured state. This process can be carried out in a statistically optimal fashion by combining each individual signal based on its reliability using a Bayesian framework. This is of importance not only for appropriate sensory-motor function in the brain, but is also crucial for many real-world industrial applications such as pose estimation for clinical movement analysis devices. One largely underexplored problem in multi-sensory integration is the role of reference frame transformations (RFTs). Different sensory signals are represented in different reference frames, e.g. sound relative to our head but vision relative to our eyes, and in order to combine this information, these signals must undergo a RFT so that they are represented in the same coordinates. This RFT requires that the brain estimates the rotation angles and axes. And since these are essentially sensory or internal signal estimates, they are noisy and have trial-to-trial variability (known as stochastic variability). Thus, a number of questions arise that we will attempt to answer: First and most important, how do noisy (stochastic) RFTs work and how do they affect multi-sensory integration in 3D? How can they be described mathematically in an analytical fashion suitable for quantitative data analyses? What brain areas carry out multi-sensory integration across reference frames? And: how can the new mathematical framework be applied to solving real-world problems for commercial movement analysis devices?We propose to answer these questions using a multi-disciplinary approach (mathematical modeling, behavioural experiments and brain imaging) that is deeply rooted in our previous NSERC-funded research on noise-less 3D RFTs. We will collaborate with C-Motion, Inc. to solve closely related multi-sensory integration problems in the pose estimation industry.