The multisensory cocktail party problem in adults: Perceptual segregation of talking faces on the basis of audiovisual temporal synchrony
The multisensory cocktail party problem in adults: Perceptual segregation of talking faces on the basis of audiovisual temporal synchrony
复制标题
DOI：
10.1016/j.cognition.2021.104743
复制
发表时间：
2021-04
期刊：
影响因子：
3.4
通讯作者：
D. Lewkowicz;M. Schmuckler;Vishakha Agrawal
中科院分区：
文献类型：
--
作者：
D. Lewkowicz;M. Schmuckler;Vishakha Agrawal
社交互动通常涉及由多个说话面孔组成的混乱的多感官场景。我们研究了视听时间同步是否能促进说话面孔的知觉分离。参与者要么看到四张相同的或四张不同的说话的脸，发出同样的可视语音的暂时抖动版本，要么听到同样的语音版本。听到的话语要么与其中一张说话的面孔发出的可见话语同步，要么与其中任何一张不同步。眼动追踪显示，参与者对同步说话的面孔表现出明显的偏好，他们盯着嘴巴的时间比盯着眼睛的时间多，他们盯着视听同步说话的面孔的眼睛的时间比盯着视听不同步说话的面孔的眼睛的时间多，当所有说话的面孔都是视听不同步的时候，他们盯着嘴巴的时间更多。这些发现表明，视听时间同步在多感官混乱的知觉分离中起着重要作用，成年人依靠说话人的眼睛和嘴巴的差异扫描策略来发现多感官连贯的来源。
Social interactions often involve a cluttered multisensory scene consisting of multiple talking faces. We investigated whether audiovisual temporal synchrony can facilitate perceptual segregation of talking faces. Participants either saw four identical or four different talking faces producing temporally jittered versions of the same visible speech utterance and heard the audible version of the same speech utterance. The audible utterance was either synchronized with the visible utterance produced by one of the talking faces or not synchronized with any of them. Eye tracking indicated that participants exhibited a marked preference for the synchronized talking face, that they gazed more at the mouth than the eyes overall, that they gazed more at the eyes of an audiovisually synchronized than a desynchronized talking face, and that they gazed more at the mouth when all talking faces were audiovisually desynchronized. These findings demonstrate that audiovisual temporal synchrony plays a major role in perceptual segregation of multisensory clutter and that adults rely on differential scanning strategies of a talker's eyes and mouth to discover sources of multisensory coherence.