CRI: CI-P: Creating the Largest Speech Emotional Database by Leveraging Existing Naturalistic Recordings

这个社区基础设施规划项目旨在考虑其他研究人员在设计最大的公开可用的自然语言情感数据库时的需求，扩大语料库在语音处理领域的影响。该项目包括与具有相关但不同专业知识的研究人员举行研讨会，介绍目前的数据收集方案，并请他们提出改进建议。拟议的活动将改进协议，以满足社区的需求。情感计算是一个旨在理解、分析、识别和综合人类情感的重要研究领域。为当前基于语音的界面提供情感功能可以促进人机交互（HCI）、医疗保健、安全和国防、教育和娱乐等领域的变革性应用。本项目设想的研究基础设施将带来新的机会，这是我们目前的语音情感数据库无法解决的。在情感计算领域，提出的语料库将提供合适的训练集来探索功能强大但需要大量标记数据的学习算法。预计所提议语料库的大小、自然度以及说话人和录音的多样性将允许社区创建跨应用程序的健壮模型。语音情感识别系统的改进将促进这些算法向实际应用的过渡，提供独特的社会效益。拟议的基础设施也将在其他语音处理任务中发挥关键作用。该社区将首次拥有解决说话人验证和自动语音识别解决方案的基础设施，以防止因情绪而引起的变化。所提出的基础架构依赖于一种基于情感检索的新方法以及基于众包的注释，以有效地构建一个大型的、自然的情感数据库，该数据库具有平衡的情感内容，降低了成本，减少了人工劳动。该数据库考虑了音频共享网站上可用的播客录音。虽然以前已经探索过使用媒体内容构建情感数据库的方法，但本研究的贡献在于使用机器学习算法来检索具有平衡情感内容的音频片段，提供具有更广泛情感的自然刺激。该方法依赖于自动算法对播客进行后处理和成本有效的注释过程，这使得建立大规模的语音情感数据库成为可能。这种方法提供了难以通过其他数据收集协议获得的自然情感再现。本项目从语料库的设计开始就涉及到社区的研究，这是本社区基础设施规划项目的关键目标。社区在选择要进行情感注释的目标句子方面也发挥了关键作用，其目标是在不受约束、未标记的录音中识别和检索目标情感行为，这是一个新的重大挑战。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This community infrastructure planning project aims to consider the needs from other researchers in the design of the largest publicly available naturalistic speech emotional database, broadening the impact of the corpus across speech processing areas. The project includes a workshop with researchers with relevant but diverse expertise to introduce the current protocol for data collection, and requests their recommendations for improvements. The proposed activity will improve the protocol to address the needs from the community. Affective computing is an important research area aiming to understand, analyze, recognize, and synthesize human emotions. Providing emotion capabilities to current speech-based interfaces can facilitate transformative applications in areas related to Human Computer Interaction (HCI), healthcare, security and defense, education and entertainment. The research infrastructure envisioned in this project will open new opportunities that we cannot address with current speech emotional databases. In the area of affective computing, the proposed corpus will provide suitable training sets to explore learning algorithms that are powerful, but require large amount of labeled data. It is expected that the size, naturalness, and speaker and recording variety in the proposed corpus will allow the community to create robust models that generalize across applications. Improvements on speech emotion recognition systems will facilitate the transition of these algorithms into practical applications, providing unique societal benefits. The proposed infrastructure will also play a key role on other speech processing tasks. For the first time, the community will have the infrastructure to address speaker verification and automatic speech recognition solutions against variations due to emotion.The proposed infrastructure relies on a novel approach based on emotion retrieval along with crowdsource-based annotations to effectively build a large, naturalistic emotional database with balanced emotional content, reduced cost and reduced manual labor. The database considers podcast recordings that are available in audio-sharing websites. Although the approach of building affective databases using media content has been previously explored, the contribution of this study is the use of machine learning algorithms to retrieve audio clips with balanced emotional content, providing natural stimuli with wider spectrum of emotions. The proposed approach relies on automatic algorithms to post-process podcasts and a cost effective annotation process, which make it possible to build large scale speech emotional databases. This approach provides natural emotional renditions that are difficult to obtain with alternative data collection protocols. This project involves the research community from the design of the corpus, which is the key goal in this community infrastructure planning project. The community also play a key role in the selection of target sentences to be emotionally annotated, with novel grand challenges where the goal is to recognize and retrieve target emotional behaviors in unconstrained, unlabeled recordings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.