Spatial Interpretation of Pointing Gestures

点击翻译按钮获取中文摘要

When we want to direct the attention of another person to a specific location or object, we often use pointing gestures. Pointing gestures are ubiquitously in the communication between humans and get increasingly relevant in human-computer and human-robot interaction. They facilitate interactions by complementing or replacing speech when it is difficult to indicate objects verbally (e.g. pointing at a specific star in the night sky). To understand such pointing gestures, an observer has to determine - among other things - which spatial location a pointer wants to indicate. Interestingly, how people extract the pointed-at position from another person's pointing gesture is not well understood. In recent work, we advanced our understanding by identifying two important factors that thwarted progress in previous studies. First, we showed that the pointer's posture needs to be precisely controlled. By contrast, it is insufficient to measure only the discrepancy between the position at which a (naïve) pointer points and the position identified by an observer, as has often been done in previous studies. Second, albeit observers may attempt to extrapolate a line from the pointer's arm and finger, their extrapolations are actually non-linear. We explained the non-linearity with a model that construes the extrapolation as the Bayesian-optimal integration of linear extrapolation and prior assumptions about likely referent positions. This model revealed a highly lawful relationship between a pointer's posture and an observer's spatial interpretation of the pointing act.With the proposed project, we systematically examine how important variables affect the spatial interpretation of pointing gestures in a highly controlled virtual reality environment. We expect that this enables to predict accurately how a given pointing gestures will be interpreted by a specific observer and which pointing gestures is best suited to single out a specific object or location.Nine experiments will be conducted that address the following questions. The first question is how the relevant spatial dimensions affect how pointing gestures are interpreted. The second question addresses how the observer's view-point affects the interpretation of pointing gestures. As a third question, we address how non-canonical pointing gestures are interpreted, in which the arm and finger are not aligned. As a forth question, we address how the spatial interpretation of a pointing gesture leads to the identification of one object and how object properties (size, relevance) affect this process. Fifth, we test whether the processes identified in highly controlled virtual reality settings are also reflected in more natural communication between humans. The results may be applied to improve embodied interaction in real-life, virtual or robotic systems, and provide a testbed for basic mechanisms of human perception.