CRI: CI-P: Creating the Largest Speech Emotional Database by Leveraging Existing Naturalistic Recordings

这个社区基础设施规划项目旨在考虑其他研究人员的需求，设计最大的公开的自然主义语音情感数据库，扩大语料库在语音处理领域的影响。该项目包括与具有相关但不同专业知识的研究人员举行一次研讨会，介绍目前的数据收集协议，并请他们提出改进建议。拟议的活动将改进协议，以满足社区的需求。情感计算是一个重要的研究领域，旨在理解，分析，识别和合成人类的情感。为当前基于语音的界面提供情感功能可以促进人机交互（HCI）、医疗保健、安全和国防、教育和娱乐等领域的变革性应用。该项目中设想的研究基础设施将开辟新的机会，我们无法解决与当前的语音情感数据库。在情感计算领域，建议的语料库将提供合适的训练集，以探索强大的学习算法，但需要大量的标记数据。预计建议语料库的大小，自然度，扬声器和录音的多样性将允许社区创建跨应用程序推广的强大模型。语音情感识别系统的改进将促进这些算法向实际应用的过渡，提供独特的社会效益。拟议的基础设施也将在其他语音处理任务中发挥关键作用。该社区将首次拥有解决说话人验证和自动语音识别解决方案的基础设施，以应对因情感而产生的变化。拟议的基础设施依赖于一种基于情感检索的新方法，沿着基于众包的注释，以有效地构建一个具有平衡情感内容的大型自然情感数据库，降低成本并减少人工劳动。该数据库考虑了音频共享网站上提供的播客录音。虽然之前已经探索了使用媒体内容构建情感数据库的方法，但本研究的贡献是使用机器学习算法来检索具有平衡情感内容的音频片段，提供具有更广泛情感的自然刺激。所提出的方法依赖于自动算法来后处理播客和成本效益的注释过程，这使得建立大规模的语音情感数据库成为可能。这种方法提供了难以用替代数据收集协议获得的自然情感再现。本项目从语料库的设计涉及到社区的研究，这是本社区基础设施规划项目的关键目标。该社区在选择要进行情感注释的目标句子方面也发挥着关键作用，其目标是在不受约束、无标签的录音中识别和检索目标情感行为。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This community infrastructure planning project aims to consider the needs from other researchers in the design of the largest publicly available naturalistic speech emotional database, broadening the impact of the corpus across speech processing areas. The project includes a workshop with researchers with relevant but diverse expertise to introduce the current protocol for data collection, and requests their recommendations for improvements. The proposed activity will improve the protocol to address the needs from the community. Affective computing is an important research area aiming to understand, analyze, recognize, and synthesize human emotions. Providing emotion capabilities to current speech-based interfaces can facilitate transformative applications in areas related to Human Computer Interaction (HCI), healthcare, security and defense, education and entertainment. The research infrastructure envisioned in this project will open new opportunities that we cannot address with current speech emotional databases. In the area of affective computing, the proposed corpus will provide suitable training sets to explore learning algorithms that are powerful, but require large amount of labeled data. It is expected that the size, naturalness, and speaker and recording variety in the proposed corpus will allow the community to create robust models that generalize across applications. Improvements on speech emotion recognition systems will facilitate the transition of these algorithms into practical applications, providing unique societal benefits. The proposed infrastructure will also play a key role on other speech processing tasks. For the first time, the community will have the infrastructure to address speaker verification and automatic speech recognition solutions against variations due to emotion.The proposed infrastructure relies on a novel approach based on emotion retrieval along with crowdsource-based annotations to effectively build a large, naturalistic emotional database with balanced emotional content, reduced cost and reduced manual labor. The database considers podcast recordings that are available in audio-sharing websites. Although the approach of building affective databases using media content has been previously explored, the contribution of this study is the use of machine learning algorithms to retrieve audio clips with balanced emotional content, providing natural stimuli with wider spectrum of emotions. The proposed approach relies on automatic algorithms to post-process podcasts and a cost effective annotation process, which make it possible to build large scale speech emotional databases. This approach provides natural emotional renditions that are difficult to obtain with alternative data collection protocols. This project involves the research community from the design of the corpus, which is the key goal in this community infrastructure planning project. The community also play a key role in the selection of target sentences to be emotionally annotated, with novel grand challenges where the goal is to recognize and retrieve target emotional behaviors in unconstrained, unlabeled recordings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.