SGER: Exploring Emotional Vocal Productions Through the Use of Real-Time Magnetic Resonance Imaging

人类使用复杂的声音和视觉编排来编码和传达意图和情感。因此，理解和利用这些表达情感的元素是促进任何人类体验的关键，包括开发以人为本的通信技术。 声乐乐器是人类表达能力的核心。虽然它在说话，唱歌和其他形式的发声中的使用已经得到了很好的研究，但对声乐作品的实际表达机制还不完全了解。例如，人类使用什么声道机制来产生表达愤怒和快乐的语音？歌手是如何用不同的情感发出不同的声音的？新的技术工具，如快速磁共振成像，结合新的计算能力，如统计机器学习，提供了以以前不可能的方式深入了解、测量和建模这些过程的方法。这个探索性研究的小补助金的重点是研究表达性人类沟通的发音发声机制。 它有两个具体的近期目标。 第一个目标旨在使用实时磁共振成像的情感语音产生的实验数据收集。该目标侧重于对收集到的图像和音频数据进行试点分析，并深入了解语音机制的情感调制及其对音频信号的影响。 这项工作将提供必要的基础，详细的研究人类情绪的语音生产。该项目的智力价值在于使用新颖的方法来检查和建模情感语音生产。 它的目的是发现人类发声过程如何被调制以编码情感表达的细节，以及这些知识是如何被用来表达情感的。 可以被纳入设计的情感语音处理和生成的机器。然而，信息的表达方面在很大程度上被忽略了技术领域的主要重点放在内容上，而不是风格;人机界面在情感认知方面受到限制。人类情感言语的研究为人类语言交际研究及其应用提供了新的方向。跨学科的方法来解决这个问题，导致其广泛的影响沿着几个方面，包括创新的实验和计算方法，可以影响几个学科，包括工程，计算机科学，心理学和语言学，使用该项目作为研究生和本科生培训的工具，以及传播的新的成像数据，迄今为止还没有在科学界。

Humans use intricate vocal and visual orchestrations to encode and communicate intent and emotions. Understanding and utilizing these expressive emotional elements hence is key to facilitating any human experience, including for developing human-centered communication technologies. The vocal instrument is central to the expressive human communication capability. While its use in speaking, singing and other forms of vocalizations have been studied well, the actual expressive mechanisms of vocal productions are less completely understood. For example, what vocal tract mechanisms do humans use to produce speech sounds conveying anger versus happiness? How do singers produce different sounds with different emotions? New technology tools, such as fast magnetic resonance imaging, combined with novel computational capabilities, such as statistical machine learning, offer ways for gaining insights into, and measuring and modeling, these processes in ways that were not possible before. This Small Grant for Exploratory Research focuses on investigating the articulatory vocal production mechanisms of expressive human communication. It has two specific near term goals. The first goal aims at experimental data collection of emotional speech production using real-time magnetic resonance imaging. The goal focuses on pilot analysis of the collected data, both image and audio, and provide insights into the emotional modulation of speech mechanisms and its consequences on the audio signal. The work will provide the necessary foundation for a detailed research study on emotional human speech production. The intellectual merit of the project lies in the use of novel methods for examining and modeling emotional speech production. It aims to discover details of how the human vocal process is modulated to encode emotional expressions and how such knowledge can be incorporated in the design of emotional speech processing and generation by the machine. Expressive aspects of information however have been largely ignored in the technology realm with the primary focus thus far put on content than style; human machine interfaces are limited in their emotional cognizance. The study of emotional human speech promises new directions in human language communication research and its applications. The interdisciplinary approach to the problem leads to its broad impact along several dimensions including the innovative experimental and computational approaches that can impact several disciplines including engineering, computer science, psychology and linguistics, the use of the project as a vehicle for graduate and undergraduate training, and the dissemination of novel imaging data that is hitherto not available in the scientific community.