The joint roles of movement and simulation in vocal communication of emotion

这项拟议的项目是我们解释声音情感交流机制的研究的继续，并包含了我们在过去几年中在歌唱背景下开发的几种方法，包括运动跟踪、凝视跟踪、肌电(EMG)和脑电(EEG)。到目前为止，我们的研究已经确定了一小部分歌曲制作的普遍原则，这些原则似乎源于人类声音的限制。违反这些原则是费力的，会导致情绪唤醒的明显表现，比如夸张的头部运动。需要新的实验来理解发声努力、头部运动和面部动画的不同方面之间的关系。歌手/扬声器的头部运动和面部动画将在三维空间(向前/向后、向上/向下、左侧/右侧)跟踪，并绕三个垂直轴(俯仰、偏航、滚动)旋转。我们还发现，观察者会跟踪并模仿这些动作，这表明模拟可能是感知声音情感交流的一个重要方面。然而，我们对观察者模拟的证据仅限于对SING的视听呈现。新的实验将包括语音和歌曲的纯听觉、纯视觉和视听呈现。观察者的眼睛运动、头部运动、面部肌肉和脑电波将被检查以寻找模拟的证据。我们预计，这些实验将揭示运动和模拟在感知情感的声音交流中的联合作用的新见解，对理解交流的动力学具有显著的意义--例如，说话者/歌手如何利用他们的声音来调节观察者的参与和视觉注意的焦点。此外，对加拿大的好处包括产生可能影响通信技术发展的基本知识(例如，改进的人机交互化身)、交流技能培训，以及为情绪处理缺陷的特殊人群(例如自闭症儿童)提供治疗。

The proposed project is a continuation of our research explicating mechanisms involved with vocal communication of emotion, and embraces several methods that we have been developing over the past few years in the context of singing including movement tracking, gaze tracking, electromyography (EMG), and electroencephalography (EEG). Our research has thus far identified a small number of universal principles of song production that appear to stem from constraints of the human voice. Violation of these principles is effortful, leading to visible displays of emotional arousal such as exaggerated head movement. New experiments are required to understand the relation between different aspects of vocal effort, head movement, and facial animation. Head movement and facial animation of singers/speakers will be tracked in three-dimensional space (forward/backward, up/down, left/right) and with rotation about three perpendicular axes (pitch, yaw, roll). We have also found that observers track these movements and mimic them, suggesting that simulation may be an important aspect of perceiving vocal communication of emotion. However, our evidence of observer simulation has been limited to audio-visual presentations of song. New experiments will involve auditory-only, visual-only, and audio-visual presentations of speech and song. Observers' eye movements, head movements, facial muscles, and brain waves will be examined for evidence of simulation. We envisage that these experiments will reveal new insights about the joint role of movement and simulation in perception of vocal communication of emotion, with striking implications for understanding the dynamics of communication -- for example, how speakers/singers can use their voices to regulate engagement and the focus of visual attention in observers. Furthermore, benefits to Canada include the generation of basic knowledge that may impact the development of communication technologies (e.g., improved avatars for human-computer interaction), communication skills training, and therapies for special populations with emotion-processing deficits (e.g., autistic children).