Neuronal integration across senses: Psychophysical and computational approaches to cue integration in injured brain

本计画将探讨知觉整合的特徴与神经元基础，进而导致有意识的经验。特别是，我们感兴趣的是视觉和听觉系统之间的相互作用，导致对事件的认识。信号检测理论为研究跨感官的刺激检测提供了坚实的基础。给出一个感觉系统（例如，听觉或视觉），我们可以量化需要从背景中区分并因此被检测到的信号水平（光强度或声音幅度）。检测能力，然后表示为在这个意义上的刺激强度的概率函数。如果两个信号（例如，光和声）被组合，则总体检测概率不同于分别从每个感测所期望的概率。这种改进的性能称为冗余增益。许多计算建模方法已被用于理解冗余增益的潜在机制（例如，奥托等人，2013年）的报告。这些方法是（unisensory）证据积累框架的扩展，在这种情况下，两个感官刺激提供证据的感知决策。他们使用反应时间数据和错误率来描述处理的各个阶段，例如信息吸收，偏见，决策标准或基于感官处理的决策中受影响的非决策成分。有趣的是，上述所有这些都依赖于检测能力，而不是有意识的意识。一个重要的发现是检测和意识的分离（Weiskrantz 2009）。这在脑损伤患者中尤其相关，这些患者在检测信号时可能明显高于偶然性，同时没有对刺激的有意识体验（标记为盲视或听觉障碍）。有趣的是，重复暴露于感官刺激可导致敏感性增加和功能恢复，例如在使用阈上多感官刺激进行系统刺激后观察到这一点（Bolognini et al 2005）。如果得到证实，这种多感觉刺激可能会导致脑损伤后康复的新技术。为了研究多感觉输入对神经元可塑性的影响，拟议的项目是一项合作努力，结合了三位导师的心理物理学，计算建模和临床神经科学的专业知识。该项目将使用心理物理学调查在报告的有意识的背景下检查视听互动。我们将开发和应用与报告意识相关的计算模型，以找出处理的哪些方面可能受到多感官输入的影响。此外，我们打算招募脑损伤（中风幸存者）的参与者，主要视觉或听觉皮层区域的病变，以检查模型的预测。这些对感觉过程的神经元编码的基础研究对于形成未来康复技术的坚实基础至关重要。

This project will investigate the characteristics and neuronal underpinning for sensory integration leading to conscious experience. In particular, we are interested in interactions between visual and auditory systems leading to awareness of events. Signal Detection Theory has provided a solid ground to investigate stimulus detection across senses. Given a sensory system (e.g., audition or vision), we can quantify the level of signal that is needed (the light intensity or the sound amplitude) to be distinguished from the background and therefore to be detected. The detection capacity is then expressed as a probability function of the stimulus strength in that sense. If two signals (e.g., both light and sound) are combined, then the overall detection probability is different than that expected from each sense separately. This improved performance is termed redundancy gain. A number of computational modelling approaches have been used to understand the mechanisms underlying redundancy gains (e.g., Otto et al., 2013). These approaches are extensions of the (unisensory) evidence accumulation framework to a scenario in which two sensory stimuli provide evidence for a perceptual decision. They use reaction time data and error rates to distil the stages of processing such as information uptake, bias, decision criteria or non-decisional components that are affected in decision making based on sensory processing. Interestingly, all the above are reliant on detection capacity and not conscious awareness.An important discovery has been the dissociation of detection and awareness (Weiskrantz 2009). This is particularly relevant in patients with brain injury, who may be significantly above chance in detecting a signal whilst having no conscious experience of a stimulus (labelled blindsight, or deaf-hearing). Interestingly, repeated exposure to sensory stimuli can lead to increased sensitivity and recovery of function, which was observed for example after systematic stimulation using supra-threshold multi-sensory stimuli (Bolognini et al 2005). If confirmed, such multi-sensory stimulation could lead to new techniques for rehabilitation after brain injury.To study the influence of multi-sensory input on neuronal plasticity, the proposed project is a collaborative effort combining expertise from psychophysics, computational modelling, and clinical neuroscience of the three supervisors. The project will examine audio-visual interaction in the context of reported conscious awareness using psychophysical investigation. We will develop and apply computational models in relation to reported awareness to find out which aspects of processing could be affected by multi-sensory input. In addition, we intend to recruit participants with brain injury (stroke survivors) with lesions in either primary visual or auditory cortical areas in order to examine the model predictions. These basic investigations on neuronal encoding of sensory processes are crucial in forming a solid foundation for future rehabilitation techniques.