Collaborative Research: Dyadic Rapport within and across Cultures: Multimodal Assessment of Human-Human and Human-Computer Interaction

不同文化背景的成员在面对面交流时可能会表现得截然不同。言语和非语言行为的特定文化方面是使一种文化的成员能够在互动的间隔时间内建立和保持彼此融洽关系的信号。融洽的关系，以及对话伙伴实现融洽关系的方式，对系统地学习很重要，因为众所周知，融洽的关系可以增加目标导向的互动成功的可能性，也可以促进知识共享和学习。微妙的暗示标志着参与、认可或欣赏。在语言通道中，这些包括对词语选择、语法结构以及声音反馈的反映。同样，非语言行为的许多方面，如姿势、凝视、点头和手势，都向对话参与者和他们的观察者发出信号--参与者感觉到从属关系的程度。一个由心理学家、人类学家、语言学家和计算机科学家组成的多学科团队将检查和比较三个不同语言/文化群体的成员在自然互动中的融洽指数：海湾/伊拉克阿拉伯语、墨西哥西班牙语和美国英语文化。该项目的一个组成部分是进一步开发能够在个人内部和对话伙伴之间对同步手势和语音进行微观分析的技术。这些技术包括用于可视化、注释和分析复杂的多模式行为的计算机接口。该项目将开发自动识别融洽状态的技术，以及导致融洽中断的语言和非语言信号。然后，这些人与人交互的检查结果将被用来为“具体式会话代理”(ECA)编写行为指令集。这些是计算机生成的二维图形，外表是人类，能够在互动中表现出听众特有的一系列语言和非语言行为。该团队将在与ECA的互动中研究人类研究参与者，在ECA中操纵与建立、维持和中断融洽关系相关的行为方面，以观察对人类参与者的影响。这项比较研究的跨文化层面将使人们有可能确定哪些方面的行为对不同文化背景的人之间成功的跨文化互动和交流至关重要。

Members of different cultures may behave quite differently from one another when interacting face-to-face. Culture-specific aspects of speech and nonverbal behavior are signals that enable members of a culture to establish and maintain a sense of rapport with one another over intervals of interaction. Rapport, and the means by which conversation partners achieve it, is important to study systematically, because rapport is known to increase the likelihood of success of goal-directed interaction, and also to promote knowledge sharing and learning. Subtle cues signal engagement, endorsement, or appreciation. In the verbal channel, these include mirroring of word choices and of grammatical structures as well as vocal feedback. Similarly, many dimensions of nonverbal behavior such as posture, gaze, nods, and gesticulation, signal -- both to the conversation participants and to observers of them -- the extent to which the participants feel a sense of affiliation. A multidisciplinary team of psychologists, anthropologists, linguists, and computer scientists, will examine and compare such indices of rapport in natural interactions across members of three diverse language/cultural groups: Gulf/Iraqi Arabic-, Mexican Spanish-, and American English-speaking cultures. An integral part of this project is to further develop technologies that enable micro-analyses of synchronized gesture and speech, both within individuals and across conversation partners. These technologies include computer interfaces for visualization, annotation, and analysis of complex, multimodal behaviors. The project will develop techniques to automatically recognize states of rapport and the verbal and nonverbal signals that lead to its disruption. The results of these examinations of human-human interaction will then be used to program behavioral repertoires for 'Embodied Conversational Agents' (ECAs). These are computer-generated, two-dimensional figures, human in appearance and capable of a range of verbal and nonverbal behaviors characteristic of listeners in interactions. The team will study human research participants in interaction with the ECAs, manipulating, in the ECAs, aspects of behavior identified as related to establishment, maintenance, and disruption of rapport, so as to observe the effects on human participants. The cross-cultural dimension of this comparative study will make it possible to identify what aspects of behavior are crucial for scaffolding successful intercultural interaction and communication among people of different cultures.