From data and theory to computational models of more effective virtual human gestures

点击翻译按钮获取中文摘要

Metaphors play a central role in how people think about and convey abstract ideas. In fact, 'conveying an idea' is a form of metaphor, as if an idea is a physical object that can be physically transferred. We speak of getting ideas across to someone, of giving someone an idea, of possessing ideas (having) or grasping an idea. Research has extensively documented a wide array of metaphor use in our language and more fundamentally in our thought processesSuch metaphors are also reflected in our gesturing. Specifically, metaphoric gestures provide a physical manifestation for these metaphors. Speakers mimic holding an imaginary object to suggest an idea. The importance of the idea may be indicated by the size of this imaginary object. Speakers also use the physical space to structure their discourse. For example, contrasting differences between ideas, this versus that, may be conveyed by gestures that refer to the ideas as physical objects located in distinct locations, thus differences in location imply differences in the ideas. Particular locations in space can also derive meaning from the underlying metaphor, For example, a common metaphor for time is that events in time are points in space along a line so that, in some cultures, events located on the left are understood as being in the past while events on the right are in the future. Gestures are known to influence understanding and information recall: listeners are more likely to learn and remember an instruction when it is accompanied by gesture (Cook & Goldin-Meadow, 2006). They also impact interpersonal attitudes of listeners towards speakers, by influencing inferences about personal and social features (Goldin-Meadow & Alibali, 2013). In particular, speakers who use metaphoric gestures are judged more persuasive and competent than those who do not (Maricchiolo et al. 2009). By associating visual features to abstract concepts and laying them out in space, metaphors and metaphoric gestures enable the use of spatial reasoning mechanisms on abstract concepts and facilitate comprehension (Beaudoin- Ryan & Goldin-Meadow, 2014; Calbris, 2011; Kendon, 2004; McNeill, 2005). The focus of this research is the computational modeling of the relation between metaphor and gesture. Beyond fostering a deeper understanding of the relation between abstract concepts, metaphor and physical gesture, the main goal of this work is to improve Human-Computer Interaction (HCI). HCI research has explored how we can make the machine more human-like. Specifically, virtual humans, autonomous agents with anthropomorphic features and behaviors engage users in face-to-face interactions, using the same verbal and nonverbal behaviors as humans. Virtual human technology has proven beneficial in a wide range of applications, including virtual patients that teach doctors how to break bad news to patients, tutors that motivate students to learn, assistants to the elders and persuasive health interventions seeking behavioral change.This proposal is a multidisciplinary effort to computationally model metaphoric gestures and systematically evaluate their effect in human-virtual human interaction. We also see this work as having a significant impact on the design of social robots that can interact with people.