CRI: CI-P: Creating the Largest Speech Emotional Database by Leveraging Existing Naturalistic Recordings

CRI：CI-P：利用现有的自然录音创建最大的语音情感数据库

• 批准号: 1823166
• 负责人: Carlos Busso
• 金额: $ 9.94万
• 依托单位: University of Texas at Dallas
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2021-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1823166&HistoricalAwards=false
• 关键词: CRI; CI; Creating; Largest; Speech

This community infrastructure planning project aims to consider the needs from other researchers in the design of the largest publicly available naturalistic speech emotional database, broadening the impact of the corpus across speech processing areas. The project includes a workshop with researchers with relevant but diverse expertise to introduce the current protocol for data collection, and requests their recommendations for improvements. The proposed activity will improve the protocol to address the needs from the community. Affective computing is an important research area aiming to understand, analyze, recognize, and synthesize human emotions. Providing emotion capabilities to current speech-based interfaces can facilitate transformative applications in areas related to Human Computer Interaction (HCI), healthcare, security and defense, education and entertainment. The research infrastructure envisioned in this project will open new opportunities that we cannot address with current speech emotional databases. In the area of affective computing, the proposed corpus will provide suitable training sets to explore learning algorithms that are powerful, but require large amount of labeled data. It is expected that the size, naturalness, and speaker and recording variety in the proposed corpus will allow the community to create robust models that generalize across applications. Improvements on speech emotion recognition systems will facilitate the transition of these algorithms into practical applications, providing unique societal benefits. The proposed infrastructure will also play a key role on other speech processing tasks. For the first time, the community will have the infrastructure to address speaker verification and automatic speech recognition solutions against variations due to emotion.The proposed infrastructure relies on a novel approach based on emotion retrieval along with crowdsource-based annotations to effectively build a large, naturalistic emotional database with balanced emotional content, reduced cost and reduced manual labor. The database considers podcast recordings that are available in audio-sharing websites. Although the approach of building affective databases using media content has been previously explored, the contribution of this study is the use of machine learning algorithms to retrieve audio clips with balanced emotional content, providing natural stimuli with wider spectrum of emotions. The proposed approach relies on automatic algorithms to post-process podcasts and a cost effective annotation process, which make it possible to build large scale speech emotional databases. This approach provides natural emotional renditions that are difficult to obtain with alternative data collection protocols. This project involves the research community from the design of the corpus, which is the key goal in this community infrastructure planning project. The community also play a key role in the selection of target sentences to be emotionally annotated, with novel grand challenges where the goal is to recognize and retrieve target emotional behaviors in unconstrained, unlabeled recordings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该社区基础设施计划项目旨在考虑其他研究人员在设计最大的公开自然主义语音情感数据库的设计中的需求，从而扩大了语料库在语音处理领域的影响。该项目包括与具有相关但多样化专业知识的研究人员的研讨会，以介绍当前的数据收集协议，并要求他们的建议进行改进。拟议的活动将改善解决社区需求的协议。情感计算是一个重要的研究领域，旨在理解，分析，识别和综合人类情绪。为当前的基于语音的界面提供情感功能可以促进与人类计算机互动（HCI），医疗保健，安全和防御，教育和娱乐相关领域的变革性应用。该项目中设想的研究基础架构将为我们无法通过当前的语音情绪数据库解决的新机会开放。在情感计算领域，拟议的语料库将提供合适的培训集，以探索功能强大但需要大量标记数据的学习算法。可以预期，拟议的语料库中的规模，自然性和说话者和记录多样性将使社区能够创建跨应用程序概括的强大模型。言语情感识别系统的改进将有助于这些算法转变为实用应用，从而提供独特的社会利益。拟议的基础架构还将在其他语音处理任务中发挥关键作用。社区首次将建立基础架构来解决扬声器验证和自动语音识别解决方案，以防止因情绪而变化。拟议的基础架构依赖于一种基于情绪检索的新方法，以及基于人群的注释，以有效地建立自然的情绪数据库，并构建一种自然的情绪数据库，并构建了平衡的情绪内容，并减少了手动劳动，并减少了手工劳动，并减少了手工劳动。该数据库考虑了在音频共享网站中可用的播客录音。尽管先前已经探讨了使用媒体内容构建情感数据库的方法，但这项研究的贡献是使用机器学习算法来检索具有平衡的情感内容的音频剪辑，从而提供了更广泛的情感范围的自然刺激。提出的方法依靠自动算法来进行后处理播客和经济有效的注释过程，这使得构建大规模的语音情感数据库成为可能。这种方法提供了自然的情感演绎，这些替代数据收集协议很难获得。该项目涉及研究社区，该研究社区的设计是该社区基础设施计划项目的关键目标。社区在选择目标句子的选择中也发挥着关键作用，并带有新颖的挑战，其目标是在不受限制的，无标记的录音中识别和检索目标情感行为。该奖项反映了NSF的法定任务，并通过使用该基金会的知识分子优点和广泛的影响来评估NSF的法定任务，并被视为值得的支持。

项目成果

Modeling Uncertainty in Predicting Emotional Attributes from Spontaneous Speech

通过自发言语预测情感属性的不确定性建模

• DOI: 10.1109/icassp40776.2020.9054237
• 发表时间: 2020
• 期刊: speech and signal processing (ICASSP 2020
• 作者: Sridhar, Kusha;Busso, Carlos
• 通讯作者: Busso, Carlos

Semi-Supervised Speech Emotion Recognition With Ladder Networks

• DOI: 10.1109/taslp.2020.3023632
• 发表时间: 2020-01-01
• 期刊: IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING
• 影响因子: 5.4
• 作者: Parthasarathy, Srinivas;Busso, Carlos
• 通讯作者: Busso, Carlos

An Efficient Temporal Modeling Approach for Speech Emotion Recognition by Mapping Varied Duration Sentences into Fixed Number of Chunks

• DOI: 10.21437/interspeech.2020-2636
• 发表时间: 2020-01-01
• 期刊: INTERSPEECH 2020
• 作者: Lin, Wei-Cheng;Busso, Carlos
• 通讯作者: Busso, Carlos

The MSP-Conversation Corpus

MSP-对话语料库

• DOI: 10.21437/interspeech.2020-2444
• 发表时间: 2020
• 期刊: Interspeech 2020
• 作者: Martinez-Lucas, Luz;Abdelwahab, Mohammed;Busso, Carlos
• 通讯作者: Busso, Carlos