A Visual Cortical Network for Deriving Phonological Information from Intelligible Lip Movements.
A Visual Cortical Network for Deriving Phonological Information from Intelligible Lip Movements.
复制标题
DOI：
10.1016/j.cub.2018.03.044
复制
发表时间：
2018-05-07
期刊：
影响因子：
--
通讯作者：
Weisz N
中科院分区：
文献类型：
--
作者：
Hauswald A;Lithari C;Collignon O;Leonardelli E;Weisz N
成功的唇读需要从视觉信息到语音信息的映射。最近，视觉和运动皮层与跟踪嘴唇运动有关（例如）。然而，目前尚不清楚视觉语音映射是否已经发生在视觉皮层水平，即该结构是否以功能相关的方式跟踪声音信号。为了阐明这一点，我们研究了皮层如何跟踪（即夹带）由无声嘴唇运动携带的缺失的声音语音信号。至关重要的是，我们将夹带与未听到的前向（可理解）和后向（不可理解）声学语音进行了对比。我们观察到，与未听到的后向声学语音包络相比，视觉皮层对未听到的前向声学语音包络表现出更强的夹带。支持视觉语音映射过程的概念，这种枕叶夹带的前后差异在实际观察到的嘴唇运动中并不存在。重要的是，相应的枕骨区域接收到更多的自上而下的输入，特别是来自左前运动区、初级运动区和体感区域，在较小程度上也来自后颞皮层。引人注目的是，在参与者中，源自这些区域的视觉皮层自上而下的调制程度与缺乏声学前向语音包络的夹带强度部分相关，但与呈现前向嘴唇运动无关。我们的研究结果表明，分布式皮质网络（包括关键的背流听觉区域）会影响视觉皮层在跟踪无声嘴唇运动时对语音清晰度的敏感性。视觉皮层比向后更好地跟踪未听到的声学语音包络 视觉与声学信号相关性引起的效果并非微不足道 在向前显示嘴唇运动期间对视觉皮层更强的自上而下控制 自上而下的影响与视觉皮层夹带效应相关 成功的唇读需要视觉语音转换。豪斯瓦尔德等人。研究表明，在处理无声播放的嘴唇运动过程中，与向后播放相比，视觉皮层会跟踪向前播放时丢失的声学语音信息。效果是自上而下控制的。
Successful lip-reading requires a mapping from visual to phonological information. Recently, visual and motor cortices have been implicated in tracking lip movements (e.g.,). It remains unclear, however, whether visuo-phonological mapping occurs already at the level of the visual cortex–that is, whether this structure tracks the acoustic signal in a functionally relevant manner. To elucidate this, we investigated how the cortex tracks (i.e., entrains to) absent acoustic speech signals carried by silent lip movements. Crucially, we contrasted the entrainment to unheard forward (intelligible) and backward (unintelligible) acoustic speech. We observed that the visual cortex exhibited stronger entrainment to the unheard forward acoustic speech envelope compared to the unheard backward acoustic speech envelope. Supporting the notion of a visuo-phonological mapping process, this forward-backward difference of occipital entrainment was not present for actually observed lip movements. Importantly, the respective occipital region received more top-down input, especially from left premotor, primary motor, and somatosensory regions and, to a lesser extent, also from posterior temporal cortex. Strikingly, across participants, the extent of top-down modulation of the visual cortex stemming from these regions partially correlated with the strength of entrainment to absent acoustic forward speech envelope, but not to present forward lip movements. Our findings demonstrate that a distributed cortical network, including key dorsal stream auditory regions, influences how the visual cortex shows sensitivity to the intelligibility of speech while tracking silent lip movements. Visual cortex tracks better forward than backward unheard acoustic speech envelope Effects not trivially caused by correlation of visual with acoustic signal Stronger top-down control of the visual cortex during forward display of lip movements Top-down influence correlates with visual cortical entrainment effect Successful lip-reading requires a visual-phonological transformation. Hauswald et al. show that, during the processing of silently played lip movements, the visual cortex tracks the missing acoustic speech information when played forward as compared to backward. The effect is under top-down control.
登录
查看更多内容
影响因子：
--
作者：
Mattout J;Henson RN;Friston KJ
通讯作者：
Friston KJ
影响因子：
15.9
作者：
Murray MM;Lewkowicz DJ;Amedi A;Wallace MT
通讯作者：
Wallace MT
影响因子：
25
作者：
通讯作者：
--
影响因子：
16.2
作者：
Michalareas G;Vezoli J;van Pelt S;Schoffelen JM;Kennedy H;Fries P
通讯作者：
Fries P
影响因子：
9.8
作者：
Gross J;Hoogenboom N;Thut G;Schyns P;Panzeri S;Belin P;Garrod S
通讯作者：
Garrod S