RI: Small: Emotional Speech Production: Analysis, Modeling and Synthesis

RI：小：情感语音生成：分析、建模和综合

• 批准号: 1116076
• 负责人: Sungbok Lee
• 金额: $ 45万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2016-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1116076&HistoricalAwards=false
• 关键词: RI; Small; Emotional; Speech; Production

Encoding of emotions in speech is achieved by vocal modulations that require an intricate control of human voicing and vocal tract articulation. The aim of this research is to identify and model articulatory processes for emotional speech production based on advanced speech production data acquisition technologies including Electromagnetic articulography (EMA) and a real-time Magnetic resonance imaging (rt-MRI). The research focuses on directly measuring and modeling articulatory kinematics and their interplay with prosodic modulation of pitch, loudness and segmental durations in speech emotion expression in order to understand the emotional speech production strategies across emotion types as well as across speakers. The validity of the emotional speech production models is verified by using a software articulatory synthesizer in an analysis-by-synthesis fashion. Theoretical implications of the findings are interpreted in relation to the Hyper and Hypo theory and the Converter/Distributor (C/D) model of speech production.Detailed knowledge on the effects of emotion on the human speech articulatory and prosodic patterning has transformative potential in developing improved speech processing technologies for emotional speech recognition and synthesis that are critical for the development of natural and robust man-machine interfaces. This goal also critically includes informing quantitative assessment of expressive speech to characterize atypical or distressed vocal behavior in diverse populations, for instance, children with Autism Spectrum Disorder (ASD). Finally, a natural by-product of this research effort is the unique articulatory database that will be shared freely with the community for further expansion of the knowledge of human speech production.

语言中的情感编码是通过声音调节来实现的，这需要对人类发声和声道的发音进行复杂的控制。本研究的目的是基于先进的语音产生数据采集技术，包括电磁发音术（EMA）和实时磁共振成像（rt-MRI），识别和模拟情绪语音产生的发音过程。研究的重点是直接测量和建模语音情感表达中的发音运动学及其与音高、响度和音段持续时间的韵律调节的相互作用，以了解不同情绪类型和不同说话者的情绪言语产生策略。利用软件发音合成器以逐合成分析的方式验证了情感语音产生模型的有效性。研究结果的理论含义是根据Hyper和Hypo理论以及语音生成的转换/分配（C/D）模型来解释的。关于情绪对人类语音发音和韵律模式的影响的详细知识在开发用于情感语音识别和合成的改进语音处理技术方面具有变革潜力，这对于开发自然和强大的人机界面至关重要。这一目标还包括为表达性言语的定量评估提供信息，以表征不同人群的非典型或痛苦发声行为，例如自闭症谱系障碍（ASD）儿童。最后，这项研究工作的自然副产品是独特的发音数据库，将与社区免费共享，以进一步扩展人类语音产生的知识。