Multi-sensory integration across reference frames

多感觉整合是将多个冗余或部分冗余的感觉信息组合在一起，以提供对测量状态的更好估计的过程。这一过程可以通过使用贝叶斯框架基于其可靠性组合每个单独的信号来以统计最优的方式进行。这不仅对大脑中适当的感觉-运动功能非常重要，而且对于许多现实世界的工业应用也是至关重要的，例如临床运动分析设备的姿势估计。在多感觉整合中，一个很少被研究的问题是参照系转换(RFTS)的作用。不同的感觉信号在不同的参考系中表示，例如相对于我们头部的声音，但相对于我们眼睛的视觉，并且为了组合该信息，这些信号必须经历RFT，以便它们在相同的坐标中表示。这种快速傅里叶变换要求大脑估计旋转角度和轴。由于这些基本上是感官或内部信号估计，它们是有噪声的，并具有试验到试验的可变性(称为随机可变性)。因此，出现了一些我们将试图回答的问题：首先，也是最重要的，噪声(随机)RFTS是如何工作的，它们如何影响3D中的多感觉整合？如何以一种适合定量数据分析的分析方式对它们进行数学描述？大脑的哪些区域进行跨参照系的多感觉整合？和：如何将新的数学框架应用于解决商业运动分析设备的现实问题？我们建议使用多学科方法(数学建模、行为实验和脑成像)来回答这些问题，这种方法深深植根于我们之前由NSERC资助的无噪声3D RFT研究。我们将与C-Motion，Inc.合作，解决姿势估计行业中密切相关的多传感器集成问题。

Multi-sensory integration is the process by which multiple redundant or partially redundant pieces of sensory information are combined in order to provide a better estimate of a measured state. This process can be carried out in a statistically optimal fashion by combining each individual signal based on its reliability using a Bayesian framework. This is of importance not only for appropriate sensory-motor function in the brain, but is also crucial for many real-world industrial applications such as pose estimation for clinical movement analysis devices. One largely underexplored problem in multi-sensory integration is the role of reference frame transformations (RFTs). Different sensory signals are represented in different reference frames, e.g. sound relative to our head but vision relative to our eyes, and in order to combine this information, these signals must undergo a RFT so that they are represented in the same coordinates. This RFT requires that the brain estimates the rotation angles and axes. And since these are essentially sensory or internal signal estimates, they are noisy and have trial-to-trial variability (known as stochastic variability). Thus, a number of questions arise that we will attempt to answer: First and most important, how do noisy (stochastic) RFTs work and how do they affect multi-sensory integration in 3D? How can they be described mathematically in an analytical fashion suitable for quantitative data analyses? What brain areas carry out multi-sensory integration across reference frames? And: how can the new mathematical framework be applied to solving real-world problems for commercial movement analysis devices?We propose to answer these questions using a multi-disciplinary approach (mathematical modeling, behavioural experiments and brain imaging) that is deeply rooted in our previous NSERC-funded research on noise-less 3D RFTs. We will collaborate with C-Motion, Inc. to solve closely related multi-sensory integration problems in the pose estimation industry.