The role of multimodal turn-completion cues in facilitating precision-timed turn transition

人类面对面互动中的话轮转换非常精确：不同语言和文化的话轮转换发生在仅200毫秒的时间间隔内。这种精确度具有符号学意义：更长的差距被视为不受欢迎的反应的先兆。这种精确的计时是如何实现的？心理语言学家一致认为，这是接受者早期预测性理解和说话者使用话轮完成提示的协同作用，包括例如，演讲形式、话轮结尾的拖拉或说话者的凝视返回接受者，给接受者最终的启动反应的‘去信号’。大量的言语、手势和韵律语篇完成线索已经被识别，尽管这些线索是基于非常小的和专门的数据集和/或在实验环境中并且很大程度上是彼此孤立的。因此，我们不知道它们是如何相互作用的，它们中的哪些对轮流过渡的精确度有实际影响，影响有多大，以及在什么条件下有效。该项目解决了这一差距。它不仅涵盖了单独的线索，而且还涵盖了面对面互动中使用的话轮完成线索的全部曲目。它的目的是确定哪些话轮完成线索被使用以及在什么条件下使用，检测它们在语境中的潜在交互作用，并确定哪些话轮完成线索对先前研究中观察到的话轮转换的精确计时产生影响。这些分析将基于弗莱堡多模式互动语料库，该语料库目前正在建设中，由德国政府提供资金(项目号414153266；参看。Https://gepris.dfg.de/gepris/projekt/414153266)和明确的多模式：一旦在2022年秋季完成，FreMIC将包括超过70个小时的视频记录，补充所有新兵的眼球跟踪视频和完整的对话分析转录；这些丰富的多模式数据将使用预测建模技术进行分析。这一综合的研究方法还建议不同专业的研究人员合作，各自贡献自己的专业知识，即Peter Auer(互动语言学，重点是凝视研究)，Christoph Rühlemann(互动语料库语言学，重点是讲故事研究)，Marion Schulte(语音学，重点是社会语音学)，Stefan Th。Gries(语料库语言学，专注于统计建模)和Judith Holler作为紧密结合的墨卡托研究员(交互语言学和心理语言学，专注于手势研究)。全面的方法、创新的数据库和该领域专家的密切合作有望大大加深我们对人类多式联运的精确计时的理解。

Turn-taking in human face-to-face interaction is remarkably precision-timed: turn transitions happen within a gap of just 200 ms across languages and cultures. The precision carries semiotic significance: longer gaps are taken as harbingers of dispreferred responses. How is this precision-timing brought about? Psycholinguists agree that it is facilitated by a synergy of early predictive comprehension by the recipient and the use of turn-completion cues by the speaker, including, for example, address forms, turn-final drawling or the return of the speaker’s gaze to the recipient, giving the recipient the ultimate ‘go-signal’ to launch the response. A large number of verbal, gestural, and prosodic turn-completion cues has already been identified, albeit based on based on very small and specialized data sets and/or in experimental settings and largely in isolation from one another. We therefore do not know how they interact, which of them have an actual effect on the precision-timing in turn transitions, how large the effect is, and under what conditions the effect holds. This project addresses this gap. It comprehensively covers not just isolated cues but the whole repertoire of turn-completion cues used in face-to-face interaction. Its objectives are to establish which turn-completion cues are used and under which conditions they are used, detect their potential interactions in context, and determine which turn-completion cues have an effect on the precision-timing of turn transition observed in prior research. The analyses will be based on the Freiburg Multimodal Interaction Corpus, currently under construction with funding from the DFG (project number 414153266; cf. https://gepris.dfg.de/gepris/projekt/414153266) and decidedly multimodal: once completed in fall 2022, FreMIC will comprise more than 70 hrs of video-recordings supplemented by eye-tracking videos for all recruits and complete conversation-analytic transcription; this rich multimodal data will be analyzed with predictive modeling techniques. This comprehensive approach also suggests a collaboration of researchers with different specializations, each contributing their specific expertise, namely Peter Auer (Interactional Linguistics, with a focus on gaze research), Christoph Rühlemann (Interactional Corpus Linguistics, with a focus on storytelling research, Marion Schulte (Phonetics, with a focus on Sociophonetics), Stefan Th. Gries (Corpus Linguistics, with a focus on statistical modeling), and Judith Holler as a closely integrated Mercator Fellow (Interactional Linguistics and Psycholinguistics, with a focus on gesture research). The comprehensive approach, the innovative database, and the close collaboration of experts in the field promise to substantially deepen our understanding of the precision timing of human multimodal communication.