Multisensory cues for conversational coordination

对话的主要单位是转身。当两个或两个以上的人交谈时，一个人发言，然后几乎无缝地轮到另一个人开始。轮转时间通常很短；在许多研究中，模态时间小于200 ms，在此值附近变化很小。这种时间分布在广泛的语言和上下文中是一致的。当考虑到5000多种现存语言的语言结构多样性时，这一事实是值得注意的。轮流制在非人类灵长类动物和鸟类中进行了研究，并发现了相似之处。此外，轮流被认为是语言和交流发展的一个关键方面。这些事实表明，回合交换的协调性是语言的一个基本方面。尽管这种行为在许多通信系统中占据首要地位，但人们对完成这种行为所需的行为和神经表征知之甚少。在拟议的研究中采取的方法是量化自然情境中的会话行为，并使用创新的方法来操纵交流的各个方面。拟议的研究包括监测纯听觉和视听对话。此外，独特的感知和眼动追踪研究将解决哪些信息有助于快速转向交换。最后，我们的研究将使用独特的方法来产生会话动态的实时扰动。这些方法包括映射说话和倾听之间的延迟函数以及使用实时动画来修改谈话者在谈话中显示的信息。通过强调对话的丰富动态，研究将集中在影响我们最常见语言使用的时间限制上。时间压力和认知资源共享如何决定语言使用的形式和能力，只有通过对话研究才能真正研究。这项研究所需的方法包括非侵入性运动和对个人自然交谈的声学测量。动画和眼动仪的新应用将加强这一领域的研究。理解对话在工程和基础科学上都很重要。虚拟形象和会话代理的使用正在迅速扩大，而现实主义在这些代理中的重要性是显而易见的。这些研究对语言处理方法和语言表征的研究问题做出了新的贡献。此外，这些研究为理解一些临床沟通障碍提供了工具和统计基础。

The primary unit of conversation is the turn. When two or more people converse, one speaker holds the floor and then almost seamlessly another person's turn begins. The turn exchange times are usually very short; the modal time in numerous studies is less than 200 ms with tight variation around this value. Such timing distributions are consistent across a wide range of languages and contexts. This fact is remarkable when the diversity of linguistic structures for the more than 5000 living languages is considered. Turn taking has been investigated in non-human primates and birds and similarities have been noted. Further, turn taking has been identified as a critical aspect of the development of language and communication. These facts suggest that the coordination of turn exchanges is a fundamental aspect of language. In spite of the primacy of this behaviour in many communication systems, much is unknown about the behavioural and neural representations needed to accomplish it. The approach taken in the proposed research is to quantify conversational behaviour in naturalistic situations and use innovative methods to manipulate aspects of the exchanges. The proposed studies include monitoring auditory-only and audiovisual conversations. In addition, unique perceptual and eye tracking studies will address what information aids rapid turn exchanges. Finally, our studies will use unique methods to produce real time perturbations of conversational dynamics. These methods include mapping the delay function between talking and listening and using real-time animation to modify the information that talkers display in conversation. By emphasizing the rich dynamics of conversation, the research will focus on the temporal constraints that shape our most common use of language. How time pressure and cognitive resource sharing determine the form and capacity of language use can only be truly studied by research about conversation. The methods that are required for this research involve non-invasive motion and acoustic measures of individuals talking naturally. New uses of animation and eye trackers will enhance research in this area. Understanding conversation has both engineering and basic science importance. The use of avatars and conversational agents is expanding rapidly and the importance of realism in these agents is obvious. The studies make novel contributions to language processing methods and research questions about language representation. Further, the studies provide the tools and statistical basis for understanding a number of clinical disorders of communication.