The joint roles of movement and simulation in vocal communication of emotion

点击翻译按钮获取中文摘要

The proposed project is a continuation of our research explicating mechanisms involved with vocal communication of emotion, and embraces several methods that we have been developing over the past few years in the context of singing including movement tracking, gaze tracking, electromyography (EMG), and electroencephalography (EEG). Our research has thus far identified a small number of universal principles of song production that appear to stem from constraints of the human voice. Violation of these principles is effortful, leading to visible displays of emotional arousal such as exaggerated head movement. New experiments are required to understand the relation between different aspects of vocal effort, head movement, and facial animation. Head movement and facial animation of singers/speakers will be tracked in three-dimensional space (forward/backward, up/down, left/right) and with rotation about three perpendicular axes (pitch, yaw, roll). We have also found that observers track these movements and mimic them, suggesting that simulation may be an important aspect of perceiving vocal communication of emotion. However, our evidence of observer simulation has been limited to audio-visual presentations of song. New experiments will involve auditory-only, visual-only, and audio-visual presentations of speech and song. Observers' eye movements, head movements, facial muscles, and brain waves will be examined for evidence of simulation. We envisage that these experiments will reveal new insights about the joint role of movement and simulation in perception of vocal communication of emotion, with striking implications for understanding the dynamics of communication -- for example, how speakers/singers can use their voices to regulate engagement and the focus of visual attention in observers. Furthermore, benefits to Canada include the generation of basic knowledge that may impact the development of communication technologies (e.g., improved avatars for human-computer interaction), communication skills training, and therapies for special populations with emotion-processing deficits (e.g., autistic children).