Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions.
Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions.
复制标题
DOI：
10.1016/j.neuroimage.2021.118841
复制
发表时间：
2022-02-15
期刊：
影响因子：
5.7
通讯作者：
Delis, Ioannis
中科院分区：
文献类型：
--
作者：
Bolam, Joshua;Boyle, Stephanie C.;Ince, Robin A. A.;Delis, Ioannis
关键词：
当我们接触到跨感觉模态的信息互补特征时，我们的大脑会在分别呈现给多个模态的刺激特征之间形成跨模态关联。例如，听觉音高-视觉尺寸关联将高音音调与小尺寸视觉对象映射，并且将低音音调与大尺寸视觉对象映射。偏好或一致的跨模态关联已被证明会影响行为表现，即跨多感官决策范式的选择准确性和反应时间（RT）。然而，这种影响知觉决策形成的神经机制仍然不清楚。在这里，我们试图确定在决策形成过程中，大脑中联想一致性的感知改善何时出现。特别是，我们问这样的改善是否代表“早期”的感觉处理的好处，或“后期”的后感官变化的决策动态。使用修改后的版本的内隐联想测试（IAT），再加上脑电图（EEG），我们测量的神经活动的听觉刺激驱动的音高大小协会的知觉决策形成的影响。行为的结果表明，参与者的反应显着更快，在审判时，听觉音高是一致的，而不是不一致的，其关联的视觉尺寸对应。我们使用多变量线性判别分析（LDA）来描述支持IAT表现的EEG活动的时空动态。我们发现了一个“早期”成分（刺激后100 - 110 ms）与听觉刺激的最大辨别力的时间相一致，和一个“晚期”成分（刺激后330 - 340 ms）潜在的IAT性能。为了验证这些组件在决策形成中的功能作用，我们采用了神经信息分层漂移扩散模型（HDDM），揭示了晚期组件降低了反应谨慎性，需要更少的感官证据来积累，而早期组件增加了不一致试验的感官编码过程的持续时间。总体而言，我们的结果提供了对“早期”感觉处理以及联想一致性的“晚期”感觉后神经表征对感知决策形成的贡献的机械见解。
When exposed to complementary features of information across sensory modalities, our brains formulate cross-modal associations between features of stimuli presented separately to multiple modalities. For example, auditory pitch-visual size associations map high-pitch tones with small-size visual objects, and low-pitch tones with large-size visual objects. Preferential, or congruent, cross-modal associations have been shown to affect behavioural performance, i.e. choice accuracy and reaction time (RT) across multisensory decision-making paradigms. However, the neural mechanisms underpinning such influences in perceptual decision formation remain unclear. Here, we sought to identify when perceptual improvements from associative congruency emerge in the brain during decision formation. In particular, we asked whether such improvements represent ‘early’ sensory processing benefits, or ‘late’ post-sensory changes in decision dynamics. Using a modified version of the Implicit Association Test (IAT), coupled with electroencephalography (EEG), we measured the neural activity underlying the effect of auditory stimulus-driven pitch-size associations on perceptual decision formation. Behavioural results showed that participants responded significantly faster during trials when auditory pitch was congruent, rather than incongruent, with its associative visual size counterpart. We used multivariate Linear Discriminant Analysis (LDA) to characterise the spatiotemporal dynamics of EEG activity underpinning IAT performance. We found an ‘Early’ component (∼100–110 ms post-stimulus onset) coinciding with the time of maximal discrimination of the auditory stimuli, and a ‘Late’ component (∼330–340 ms post-stimulus onset) underlying IAT performance. To characterise the functional role of these components in decision formation, we incorporated a neurally-informed Hierarchical Drift Diffusion Model (HDDM), revealing that the Late component decreases response caution, requiring less sensory evidence to be accumulated, whereas the Early component increased the duration of sensory-encoding processes for incongruent trials. Overall, our results provide a mechanistic insight into the contribution of ‘early’ sensory processing, as well as ‘late’ post-sensory neural representations of associative congruency to perceptual decision formation.
登录
查看更多内容
DOI：
10.1111/ejn.13724
发表时间：
2017-11
期刊：
The European journal of neuroscience
影响因子：
--
作者：
Boyle SC;Kayser SJ;Kayser C
通讯作者：
Kayser C
影响因子：
5.7
作者：
Delis I;Dmochowski JP;Sajda P;Wang Q
通讯作者：
Wang Q
影响因子：
5.3
作者：
Baier, Bernhard;Kleinschmidt, Andreas;Mueller, Notger G.
通讯作者：
Mueller, Notger G.
影响因子：
5.3
作者：
Dahl, Christoph D.;Logothetis, Nikos K.;Kayser, Christoph
通讯作者：
Kayser, Christoph
影响因子：
4.6
作者：
Bang JW;Rahnev D
通讯作者：
Rahnev D