Multisensory cues for conversational coordination

对话协调的多感官线索

• 批准号: RGPIN-2019-04997
• 负责人: Munhall, Kevin
• 金额: $ 3.92万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=714918
• 关键词: Multisensory; cues; conversational; coordination

The primary unit of conversation is the turn. When two or more people converse, one speaker holds the floor and then almost seamlessly another person's turn begins. The turn exchange times are usually very short; the modal time in numerous studies is less than 200 ms with tight variation around this value. Such timing distributions are consistent across a wide range of languages and contexts. This fact is remarkable when the diversity of linguistic structures for the more than 5000 living languages is considered. Turn taking has been investigated in non-human primates and birds and similarities have been noted. Further, turn taking has been identified as a critical aspect of the development of language and communication. These facts suggest that the coordination of turn exchanges is a fundamental aspect of language. In spite of the primacy of this behaviour in many communication systems, much is unknown about the behavioural and neural representations needed to accomplish it. The approach taken in the proposed research is to quantify conversational behaviour in naturalistic situations and use innovative methods to manipulate aspects of the exchanges. The proposed studies include monitoring auditory-only and audiovisual conversations. In addition, unique perceptual and eye tracking studies will address what information aids rapid turn exchanges. Finally, our studies will use unique methods to produce real time perturbations of conversational dynamics. These methods include mapping the delay function between talking and listening and using real-time animation to modify the information that talkers display in conversation. By emphasizing the rich dynamics of conversation, the research will focus on the temporal constraints that shape our most common use of language. How time pressure and cognitive resource sharing determine the form and capacity of language use can only be truly studied by research about conversation. The methods that are required for this research involve non-invasive motion and acoustic measures of individuals talking naturally. New uses of animation and eye trackers will enhance research in this area. Understanding conversation has both engineering and basic science importance. The use of avatars and conversational agents is expanding rapidly and the importance of realism in these agents is obvious. The studies make novel contributions to language processing methods and research questions about language representation. Further, the studies provide the tools and statistical basis for understanding a number of clinical disorders of communication.

谈话的基本单位是话轮。当两个或两个以上的人匡威时，一个发言者占据发言权，然后几乎无缝地开始轮到另一个人发言。 转向交换时间通常很短;许多研究中的模态时间小于200 ms，在此值附近变化很小。这种时间分布在各种语言和上下文中是一致的。 当考虑到5000多种现存语言的语言结构的多样性时，这一事实是引人注目的。在非人类灵长类动物和鸟类中已经对轮流进行了研究，并注意到了相似之处。 此外，话轮转换被认为是语言和交际发展的一个重要方面。 这些事实表明，话轮交换的协调是语言的一个基本方面。 尽管这种行为在许多通信系统中占据首要地位，但对于实现它所需的行为和神经表征还知之甚少。拟议研究中采取的方法是量化自然情景中的对话行为，并使用创新方法来操纵交流的各个方面。拟议的研究包括监测仅录音和视听对话。 此外，独特的知觉和眼动跟踪研究将解决什么信息有助于快速转身交流。 最后，我们的研究将使用独特的方法来产生会话动态的真实的时间扰动。 这些方法包括映射说话和倾听之间的延迟函数，以及使用实时动画来修改说话者在会话中显示的信息。 通过强调对话的丰富动态，研究将集中在塑造我们最常见的语言使用的时间限制。时间压力和认知资源共享如何决定语言使用的形式和能力，只有通过对会话的研究才能真正研究。这项研究所需的方法涉及非侵入性的运动和个人自然交谈的声学测量。动画和眼动仪的新用途将加强这一领域的研究。 理解对话对工程和基础科学都很重要。 化身和会话代理的使用正在迅速扩大，这些代理中的现实主义的重要性是显而易见的。 这些研究为语言处理方法和语言表征问题的研究做出了新的贡献。 此外，这些研究为理解一些临床沟通障碍提供了工具和统计基础。