The role of multimodal turn-completion cues in facilitating precision-timed turn transition

多模式转弯完成提示在促进精确定时转弯转换中的作用

• 批准号: 497779797
• 负责人: Professor Dr. Peter Auer
• 金额: --
• 依托单位: Institut für Anglistik
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/497779797?language=en
• 关键词: role; multimodal; turn; completion; cues

Turn-taking in human face-to-face interaction is remarkably precision-timed: turn transitions happen within a gap of just 200 ms across languages and cultures. The precision carries semiotic significance: longer gaps are taken as harbingers of dispreferred responses. How is this precision-timing brought about? Psycholinguists agree that it is facilitated by a synergy of early predictive comprehension by the recipient and the use of turn-completion cues by the speaker, including, for example, address forms, turn-final drawling or the return of the speaker’s gaze to the recipient, giving the recipient the ultimate ‘go-signal’ to launch the response. A large number of verbal, gestural, and prosodic turn-completion cues has already been identified, albeit based on based on very small and specialized data sets and/or in experimental settings and largely in isolation from one another. We therefore do not know how they interact, which of them have an actual effect on the precision-timing in turn transitions, how large the effect is, and under what conditions the effect holds. This project addresses this gap. It comprehensively covers not just isolated cues but the whole repertoire of turn-completion cues used in face-to-face interaction. Its objectives are to establish which turn-completion cues are used and under which conditions they are used, detect their potential interactions in context, and determine which turn-completion cues have an effect on the precision-timing of turn transition observed in prior research. The analyses will be based on the Freiburg Multimodal Interaction Corpus, currently under construction with funding from the DFG (project number 414153266; cf. https://gepris.dfg.de/gepris/projekt/414153266) and decidedly multimodal: once completed in fall 2022, FreMIC will comprise more than 70 hrs of video-recordings supplemented by eye-tracking videos for all recruits and complete conversation-analytic transcription; this rich multimodal data will be analyzed with predictive modeling techniques. This comprehensive approach also suggests a collaboration of researchers with different specializations, each contributing their specific expertise, namely Peter Auer (Interactional Linguistics, with a focus on gaze research), Christoph Rühlemann (Interactional Corpus Linguistics, with a focus on storytelling research, Marion Schulte (Phonetics, with a focus on Sociophonetics), Stefan Th. Gries (Corpus Linguistics, with a focus on statistical modeling), and Judith Holler as a closely integrated Mercator Fellow (Interactional Linguistics and Psycholinguistics, with a focus on gesture research). The comprehensive approach, the innovative database, and the close collaboration of experts in the field promise to substantially deepen our understanding of the precision timing of human multimodal communication.

人类面对面互动中的话轮转换是非常精确的：在不同的语言和文化中，话轮转换发生在200毫秒的间隔内。这种精确性具有符号学意义：较长的间隙被视为不满意反应的预兆。这种精确的定时是如何实现的？心理语言学家一致认为，这是促进了协同效应的早期预测理解的收件人和使用轮完成线索的发言者，包括，例如，地址形式，话轮最后拖长或返回的扬声器的凝视收件人，给收件人最终的“去信号”，以启动响应。 大量的言语，手势和韵律的话轮完成线索已经被确定，尽管基于非常小的和专门的数据集和/或在实验环境中，并且在很大程度上彼此隔离。因此，我们不知道它们是如何相互作用的，它们中的哪一个对轮流跃迁的精确定时有实际影响，这种影响有多大，以及在什么条件下这种影响成立。 该项目解决了这一差距。它不仅全面涵盖了孤立的线索，而且涵盖了面对面互动中使用的完整的话轮完成线索。它的目标是确定哪些话轮完成线索被使用，在何种条件下使用，检测它们在上下文中的潜在相互作用，并确定哪些话轮完成线索对先前研究中观察到的话轮转换的精确时间有影响。 这些分析将基于弗赖堡多模式交互语料库，该语料库目前正在建设中，由DFG提供资金（项目编号414153266;参见https：//gepris.dfg.de/gepris/projekt/414153266）和决定性的多模态：一旦在2022年秋季完成，FreMIC将包括超过70小时的视频记录，辅以所有新兵的眼动跟踪视频和完整的对话分析转录;这些丰富的多模态数据将通过预测建模技术进行分析。 这种全面的方法也建议不同专业的研究人员合作，每个人都贡献自己的专业知识，即Peter奥尔（InterestingLinguistics，专注于凝视研究），Christoph Rühlemann（InterestingCorpus Linguistics，专注于讲故事研究），Marion Schulte（语音学，专注于社会语音学），Stefan Th. Gries（语料库语言学，专注于统计建模），以及Judith Holler作为密切结合的墨卡托研究员（Interestinglinguistics和Psycholinguistics，专注于手势研究）。 全面的方法，创新的数据库以及该领域专家的密切合作，有望大大加深我们对人类多模态通信精确计时的理解。