Collaborative Research: Dyadic Rapport within and across Cultures: Multimodal Assessment of Human-Human and Human-Computer Interaction

不同文化背景的人在面对面交流时可能表现得截然不同。语言和非语言行为的特定文化方面是一种信号，它使一种文化的成员能够在互动的间隔中彼此建立和保持一种融洽感。系统地研究融洽关系以及对话伙伴实现融洽关系的手段是很重要的，因为众所周知，融洽关系可以增加目标导向互动成功的可能性，也可以促进知识共享和学习。微妙的暗示暗示着参与、认可或欣赏。在言语通道中，这些包括单词选择和语法结构的镜像以及声音反馈。同样，非语言行为的许多方面，如姿势、凝视、点头和手势，都向对话参与者和他们的观察者发出信号，表明参与者感到归属感的程度。一个由心理学家、人类学家、语言学家和计算机科学家组成的多学科团队，将研究和比较来自三个不同语言/文化群体（海湾/伊拉克阿拉伯语、墨西哥西班牙语和美国英语文化）的成员在自然互动中的这种融洽指数。该项目的一个组成部分是进一步开发技术，使同步手势和语音的微观分析成为可能，无论是在个人内部还是在对话伙伴之间。这些技术包括用于可视化、注释和分析复杂、多模态行为的计算机接口。该项目将开发技术来自动识别融洽关系的状态以及导致破坏融洽关系的语言和非语言信号。这些人与人之间互动的检查结果将被用于为“具体化对话代理”（eca）编程行为库。这些是计算机生成的二维图形，具有人类的外观，能够在互动中表现出听众的一系列语言和非语言行为特征。该小组将研究人类研究参与者与eca的相互作用，在eca中操纵被确定为与建立、维持和破坏关系有关的行为方面，以便观察对人类参与者的影响。这项比较研究的跨文化维度将使我们有可能确定行为的哪些方面对于不同文化的人之间成功的跨文化互动和沟通至关重要。

Members of different cultures may behave quite differently from one another when interacting face-to-face. Culture-specific aspects of speech and nonverbal behavior are signals that enable members of a culture to establish and maintain a sense of rapport with one another over intervals of interaction. Rapport, and the means by which conversation partners achieve it, is important to study systematically, because rapport is known to increase the likelihood of success of goal-directed interaction, and also to promote knowledge sharing and learning. Subtle cues signal engagement, endorsement, or appreciation. In the verbal channel, these include mirroring of word choices and of grammatical structures as well as vocal feedback. Similarly, many dimensions of nonverbal behavior such as posture, gaze, nods, and gesticulation, signal -- both to the conversation participants and to observers of them -- the extent to which the participants feel a sense of affiliation. A multidisciplinary team of psychologists, anthropologists, linguists, and computer scientists, will examine and compare such indices of rapport in natural interactions across members of three diverse language/cultural groups: Gulf/Iraqi Arabic-, Mexican Spanish-, and American English-speaking cultures. An integral part of this project is to further develop technologies that enable micro-analyses of synchronized gesture and speech, both within individuals and across conversation partners. These technologies include computer interfaces for visualization, annotation, and analysis of complex, multimodal behaviors. The project will develop techniques to automatically recognize states of rapport and the verbal and nonverbal signals that lead to its disruption. The results of these examinations of human-human interaction will then be used to program behavioral repertoires for 'Embodied Conversational Agents' (ECAs). These are computer-generated, two-dimensional figures, human in appearance and capable of a range of verbal and nonverbal behaviors characteristic of listeners in interactions. The team will study human research participants in interaction with the ECAs, manipulating, in the ECAs, aspects of behavior identified as related to establishment, maintenance, and disruption of rapport, so as to observe the effects on human participants. The cross-cultural dimension of this comparative study will make it possible to identify what aspects of behavior are crucial for scaffolding successful intercultural interaction and communication among people of different cultures.