Collaborative Research: Dyadic Rapport within and across Cultures: Multimodal Assessment of Human-Human and Human-Computer Interaction

点击翻译按钮获取中文摘要

Members of different cultures may behave quite differently from one another when interacting face-to-face. Culture-specific aspects of speech and nonverbal behavior are signals that enable members of a culture to establish and maintain a sense of rapport with one another over intervals of interaction. Rapport, and the means by which conversation partners achieve it, is important to study systematically, because rapport is known to increase the likelihood of success of goal-directed interaction, and also to promote knowledge sharing and learning. Subtle cues signal engagement, endorsement, or appreciation. In the verbal channel, these include mirroring of word choices and of grammatical structures as well as vocal feedback. Similarly, many dimensions of nonverbal behavior such as posture, gaze, nods, and gesticulation, signal -- both to the conversation participants and to observers of them -- the extent to which the participants feel a sense of affiliation. A multidisciplinary team of psychologists, anthropologists, linguists, and computer scientists, will examine and compare such indices of rapport in natural interactions across members of three diverse language/cultural groups: Gulf/Iraqi Arabic-, Mexican Spanish-, and American English-speaking cultures. An integral part of this project is to further develop technologies that enable micro-analyses of synchronized gesture and speech, both within individuals and across conversation partners. These technologies include computer interfaces for visualization, annotation, and analysis of complex, multimodal behaviors. The project will develop techniques to automatically recognize states of rapport and the verbal and nonverbal signals that lead to its disruption. The results of these examinations of human-human interaction will then be used to program behavioral repertoires for 'Embodied Conversational Agents' (ECAs). These are computer-generated, two-dimensional figures, human in appearance and capable of a range of verbal and nonverbal behaviors characteristic of listeners in interactions. The team will study human research participants in interaction with the ECAs, manipulating, in the ECAs, aspects of behavior identified as related to establishment, maintenance, and disruption of rapport, so as to observe the effects on human participants. The cross-cultural dimension of this comparative study will make it possible to identify what aspects of behavior are crucial for scaffolding successful intercultural interaction and communication among people of different cultures.