CRI: CI-P: Creating the Largest Speech Emotional Database by Leveraging Existing Naturalistic Recordings

这个社区基础设施规划项目的目的是在设计最大的公开可用的自然语言情感数据库时考虑其他研究人员的需求，扩大语料库在语音处理领域的影响。该项目包括与具有相关但不同专门知识的研究人员举办讲习班，介绍目前的数据收集议定书，并请他们提出改进建议。拟议的活动将改进议定书，以满足社区的需求。情感计算是理解、分析、识别和合成人类情感的一个重要研究领域。向当前基于语音的界面提供情感功能可以促进与人机交互(HCI)、医疗保健、安全和国防、教育和娱乐相关领域的变革性应用。这个项目中设想的研究基础设施将打开新的机会，这些机会是我们目前的语音情感数据库无法解决的。在情感计算领域，所提出的语料库将提供合适的训练集，以探索功能强大但需要大量标记数据的学习算法。预计拟议语料库中的大小、自然度以及说话者和录音的多样性将允许社区创建跨应用程序通用的健壮模型。语音情感识别系统的改进将有助于将这些算法转化为实际应用，提供独特的社会效益。拟议的基础设施还将在其他语音处理任务中发挥关键作用。社区将首次拥有针对情绪变化的说话人验证和自动语音识别解决方案的基础设施。拟议的基础设施依赖于一种基于情感检索和基于众源注释的新方法，以有效地建立一个大型的、自然的情感数据库，具有平衡的情感内容，降低成本和减少体力劳动。该数据库考虑音频共享网站上提供的播客录音。虽然以前已经探索过使用媒体内容建立情感数据库的方法，但本研究的贡献是使用机器学习算法来检索具有平衡情感内容的音频片段，为自然刺激提供更广泛的情感范围。该方法依赖于自动算法对播客进行后处理，并采用高性价比的标注过程，这使得建立大规模的语音情感数据库成为可能。这种方法提供了自然的情感再现，这是使用替代数据收集协议难以获得的。这个项目从语料库的设计上涉及到研究社区，这是这个社区基础设施规划项目的关键目标。社区在选择要进行情感注释的目标句子方面也发挥了关键作用，面临着新的重大挑战，目标是在不受限制的、未标记的录音中识别和检索目标情感行为。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This community infrastructure planning project aims to consider the needs from other researchers in the design of the largest publicly available naturalistic speech emotional database, broadening the impact of the corpus across speech processing areas. The project includes a workshop with researchers with relevant but diverse expertise to introduce the current protocol for data collection, and requests their recommendations for improvements. The proposed activity will improve the protocol to address the needs from the community. Affective computing is an important research area aiming to understand, analyze, recognize, and synthesize human emotions. Providing emotion capabilities to current speech-based interfaces can facilitate transformative applications in areas related to Human Computer Interaction (HCI), healthcare, security and defense, education and entertainment. The research infrastructure envisioned in this project will open new opportunities that we cannot address with current speech emotional databases. In the area of affective computing, the proposed corpus will provide suitable training sets to explore learning algorithms that are powerful, but require large amount of labeled data. It is expected that the size, naturalness, and speaker and recording variety in the proposed corpus will allow the community to create robust models that generalize across applications. Improvements on speech emotion recognition systems will facilitate the transition of these algorithms into practical applications, providing unique societal benefits. The proposed infrastructure will also play a key role on other speech processing tasks. For the first time, the community will have the infrastructure to address speaker verification and automatic speech recognition solutions against variations due to emotion.The proposed infrastructure relies on a novel approach based on emotion retrieval along with crowdsource-based annotations to effectively build a large, naturalistic emotional database with balanced emotional content, reduced cost and reduced manual labor. The database considers podcast recordings that are available in audio-sharing websites. Although the approach of building affective databases using media content has been previously explored, the contribution of this study is the use of machine learning algorithms to retrieve audio clips with balanced emotional content, providing natural stimuli with wider spectrum of emotions. The proposed approach relies on automatic algorithms to post-process podcasts and a cost effective annotation process, which make it possible to build large scale speech emotional databases. This approach provides natural emotional renditions that are difficult to obtain with alternative data collection protocols. This project involves the research community from the design of the corpus, which is the key goal in this community infrastructure planning project. The community also play a key role in the selection of target sentences to be emotionally annotated, with novel grand challenges where the goal is to recognize and retrieve target emotional behaviors in unconstrained, unlabeled recordings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.