Multisensory cues for conversational coordination

对话的主要单位是话轮。当两个或两个以上的人交谈时，一个发言者保持发言，然后几乎无缝地开始另一个人的发言。话轮交换时间通常很短；许多研究中的模式时间小于200ms，在这个值附近有很大的变化。这样的时间分布在广泛的语言和上下文中是一致的。当考虑到5000多种活着的语言的语言结构的多样性时，这一事实是值得注意的。非人灵长类动物和鸟类的转弯行为已经被研究过，并注意到相似之处。此外，话轮转换被认为是语言和交际发展的一个重要方面。这些事实表明，话轮转换的协调是语言的一个基本方面。*尽管这一行为在许多通信系统中处于首要地位，但人们对实现这一行为所需的行为和神经表征仍知之甚少。这项拟议的研究采用的方法是量化自然情景下的对话行为，并使用创新的方法来操纵交流的各个方面。拟议的研究包括监测纯听觉和视听对话。此外，独特的感知和眼球跟踪研究将解决哪些信息有助于快速交流。最后，我们的研究将使用独特的方法来产生会话动态的实时扰动。这些方法包括映射说和听之间的延迟函数，以及使用实时动画来修改说话者在对话中显示的信息。*通过强调对话的丰富动态，研究将重点放在塑造我们最常见语言使用的时间约束上。只有通过对会话的研究，才能真正研究时间压力和认知资源共享如何决定语言使用的形式和能力。这项研究所需的方法包括非侵入性运动和个人自然交谈的声学测量。动画和眼球追踪器的新用途将加强这一领域的研究。理解对话具有工程学和基础科学的重要性。化身和对话代理的使用正在迅速扩大，真实感在这些代理中的重要性是显而易见的。这些研究在语言加工方法和关于语言表征的研究问题上做出了新的贡献。此外，这些研究为了解一些临床上的沟通障碍提供了工具和统计基础。**

The primary unit of conversation is the turn. When two or more people converse, one speaker holds the floor and then almost seamlessly another person's turn begins. The turn exchange times are usually very short; the modal time in numerous studies is less than 200 ms with tight variation around this value. Such timing distributions are consistent across a wide range of languages and contexts. This fact is remarkable when the diversity of linguistic structures for the more than 5000 living languages is considered. Turn taking has been investigated in non-human primates and birds and similarities have been noted. Further, turn taking has been identified as a critical aspect of the development of language and communication. These facts suggest that the coordination of turn exchanges is a fundamental aspect of language. ******In spite of the primacy of this behaviour in many communication systems, much is unknown about the behavioural and neural representations needed to accomplish it. The approach taken in the proposed research is to quantify conversational behaviour in naturalistic situations and use innovative methods to manipulate aspects of the exchanges. The proposed studies include monitoring auditory-only and audiovisual conversations. In addition, unique perceptual and eye tracking studies will address what information aids rapid turn exchanges. Finally, our studies will use unique methods to produce real time perturbations of conversational dynamics. These methods include mapping the delay function between talking and listening and using real-time animation to modify the information that talkers display in conversation.******By emphasizing the rich dynamics of conversation, the research will focus on the temporal constraints that shape our most common use of language. How time pressure and cognitive resource sharing determine the form and capacity of language use can only be truly studied by research about conversation. The methods that are required for this research involve non-invasive motion and acoustic measures of individuals talking naturally. New uses of animation and eye trackers will enhance research in this area.******Understanding conversation has both engineering and basic science importance. The use of avatars and conversational agents is expanding rapidly and the importance of realism in these agents is obvious. The studies make novel contributions to language processing methods and research questions about language representation. Further, the studies provide the tools and statistical basis for understanding a number of clinical disorders of communication. **