EAGER: Exploring the Use of Synthetic Speech as Reference Model to Detect Salient Emotional Segments in Speech

EAGER:探索使用合成语音作为参考模型来检测语音中的显着情感片段

基本信息

  • 批准号:
    1329659
  • 负责人:
  • 金额:
    $ 5.93万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2013
  • 资助国家:
    美国
  • 起止时间:
    2013-03-15 至 2014-08-31
  • 项目状态:
    已结题

项目摘要

This EArly Grant for Exploratory Research aims to create neutral reference model from synthetic speech to contrast the emotional content of a speech signal. Emotional understanding is a crucial skill in human communication. For this reason, modeling and recognizing emotions is essential in the design and implementation of interfaces that are more in tune with the user's needs. Starting from the premise that paralinguistic information is non-uniformly conveyed across time, this study aims to identify emotionally prominent regions or focal points across various acoustic features. The study explores a novel approach based on synthetic speech to build reference models characterizing patterns observed in neutral speech. These reference models are used to contrast the emotional information observed in localized segments of a speech signal. The study builds a synthetic speech signal that conveys the same lexical information and is timely aligned with the target sentence in the database. Since it is expected that a single synthetic speech will not capture the full range of variability observed in neutral speech, the study explores approaches to produce different neutral synthetic realizations. After creating a parallel corpus with time-aligned synthetic speech, the study explores how well synthetic speech captures the acoustic patterns and emotional percepts of neutral, nonemotional speech. Then, a target signal from the database is compared with the properties observed across the family of synthesized signals. The study presents a novel approach to build a robust emotion recognition system that exploits the underlying nonuniform externalization process of expressive behaviors. Algorithms that able to identify localized emotional segments have the potential to shift the current approaches used in the area of affective computing. Instead of recognizing the emotional content of pre-segmented sentences, the problem is formulated as a detection paradigm, which is appealing from an application perspective. These advances represent a transformative breakthrough in the area of behavioral analysis and affective computing. The proposed models and algorithms provide numerous insights to explore and extend theories in linguistic and paralinguistic human behavior. Having established the base infrastructure for this exploratory research, several new scientific avenues will emerge that serve as truly innovative advancements that will impact applications in security and defense, next generation of advanced user interfaces, health informatics, and education. Furthermore, the scientific methods are enriching venues for interdisciplinary training and mentoring for undergraduate and graduate students.
这项早期探索性研究的目的是从合成语音中创建中性参考模型,以对比语音信号的情感内容。情感理解是人类交流中的一项重要技能。因此,在设计和实现更符合用户需求的界面时,建模和识别情绪是必不可少的。本研究从副语言信息在时间上的非均匀传递这一前提出发,旨在通过不同的声学特征识别情绪显著区域或焦点。这项研究探索了一种基于合成语音的新方法来构建描述中性语音中观察到的模式的参考模型。这些参考模型用于对比在语音信号的局部片段中观察到的情感信息。这项研究建立了一个合成语音信号,它传达了相同的词汇信息,并与数据库中的目标句子及时对齐。由于预计单一的合成语音不能捕捉到中性语音中观察到的全部可变性,因此本研究探索了产生不同中性合成实现的方法。在创建了一个与时间一致的合成语音的平行语料库后,这项研究探索了合成语音如何很好地捕捉到中性、非情绪化语音的声学模式和情感感知。然后,将来自数据库的目标信号与在合成信号族中观察到的属性进行比较。这项研究提出了一种新的方法来建立一个健壮的情绪识别系统,该系统利用表达行为潜在的非均匀外化过程。能够识别局部情感片段的算法有可能改变目前在情感计算领域使用的方法。该问题没有识别预切分句子的情感内容,而是将其描述为一种检测范式,从应用的角度来看是很有吸引力的。这些进展代表了行为分析和情感计算领域的一项革命性突破。所提出的模型和算法为探索和扩展语言和副语言人类行为的理论提供了大量的见解。在为这项探索性研究建立了基础设施后,将出现几个新的科学途径,这些途径将成为真正的创新进步,将影响安全和国防、下一代高级用户界面、健康信息学和教育方面的应用。此外,科学的方法丰富了本科生和研究生进行跨学科培训和指导的场所。

项目成果

期刊论文数量(1)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Lexical Dependent Emotion Detection Using Synthetic Speech Reference
使用合成语音参考进行词汇相关情绪检测
  • DOI:
    10.1109/access.2019.2898353
  • 发表时间:
    2019
  • 期刊:
  • 影响因子:
    3.9
  • 作者:
    Lotfian, Reza;Busso, Carlos
  • 通讯作者:
    Busso, Carlos
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Carlos Busso其他文献

Enhanced Facial Landmarks Detection for Patients with Repaired Cleft Lip and Palate
增强唇裂和腭裂修复患者的面部标志检测
  • DOI:
  • 发表时间:
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Karen Rosero;Ali N. Salman;Berrak Sisman;R. Hallac;Carlos Busso
  • 通讯作者:
    Carlos Busso
SPEECH EMOTION RECOGNITION IN REAL STATIC AND DYNAMIC HUMAN-ROBOT INTERACTION SCENARIOS
真实静态和动态人机交互场景中的语音情感识别
  • DOI:
    10.1016/j.csl.2024.101666
  • 发表时间:
    2024
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Nicolás Grágeda;Carlos Busso;Eduardo Alvarado;Ricardo García;R. Mahú;F. Huenupán;N. B. Yoma
  • 通讯作者:
    N. B. Yoma
Mixed Emotion Modelling for Emotional Voice Conversion
用于情感语音转换的混合情感建模
  • DOI:
  • 发表时间:
    2022
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Kun Zhou;Berrak Sisman;Carlos Busso;Haizhou Li
  • 通讯作者:
    Haizhou Li
Richness and Density of Birds in Timber Nothofagus pumilio Forests and their Unproductive Associated Environments
  • DOI:
    10.1007/s10531-004-1665-0
  • 发表时间:
    2005-09-01
  • 期刊:
  • 影响因子:
    3.100
  • 作者:
    María Vanessa Lencinas;Guillermo Martínez Pastur;Marlin Medina;Carlos Busso
  • 通讯作者:
    Carlos Busso
Towards Naturalistic Voice Conversion: NaturalVoices Dataset with an Automatic Processing Pipeline
迈向自然语音转换:具有自动处理管道的 NaturalVoices 数据集
  • DOI:
  • 发表时间:
    2024
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Ali N. Salman;Zongyang Du;Shreeram Suresh Chandra;Ismail Rasim Ulgen;Carlos Busso;Berrak Sisman
  • 通讯作者:
    Berrak Sisman

Carlos Busso的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Carlos Busso', 18)}}的其他基金

CCRI: Medium: MSP-Podcast: Creating The Largest Speech Emotional Database By Leveraging Existing Naturalistic Recordings
CCRI:媒介:MSP-Podcast:利用现有的自然主义录音创建最大的语音情感数据库
  • 批准号:
    2016719
  • 财政年份:
    2020
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
CRI: CI-P: Creating the Largest Speech Emotional Database by Leveraging Existing Naturalistic Recordings
CRI:CI-P:利用现有的自然录音创建最大的语音情感数据库
  • 批准号:
    1823166
  • 财政年份:
    2018
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
RI: Small: Integrative, Semantic-Aware, Speech-Driven Models for Believable Conversational Agents with Meaningful Behaviors
RI:小型:集成的、语义感知的、语音驱动的模型,用于具有有意义行为的可信会话代理
  • 批准号:
    1718944
  • 财政年份:
    2017
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
FG 2015 Doctoral Consortium: Travel Support for Graduate Students
FG 2015 博士联盟:研究生旅行支持
  • 批准号:
    1540944
  • 财政年份:
    2015
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
CAREER: Advanced Knowledge Extraction of Affective Behaviors During Natural Human Interaction
职业:人类自然互动过程中情感行为的高级知识提取
  • 批准号:
    1453781
  • 财政年份:
    2015
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Continuing Grant
WORKSHOP: Doctoral Consortium for the International Conference on Multimodal Interaction (ICMI 2013)
研讨会:多模式交互国际会议博士联盟 (ICMI 2013)
  • 批准号:
    1346655
  • 财政年份:
    2013
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
RI: Small: Collaborative Research: Exploring Audiovisual Emotion Perception using Data-Driven Computational Modeling
RI:小型:协作研究:使用数据驱动的计算模型探索视听情感感知
  • 批准号:
    1217104
  • 财政年份:
    2012
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Continuing Grant
Workshop: Doctoral Consortium at the 14th International Conference on Multimodal Interaction
研讨会:第14届多模态交互国际会议博士联盟
  • 批准号:
    1249319
  • 财政年份:
    2012
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant

相似国自然基金

Exploring Changing Fertility Intentions in China
  • 批准号:
  • 批准年份:
    2024
  • 资助金额:
    万元
  • 项目类别:
    外国学者研究基金
Exploring the Intrinsic Mechanisms of CEO Turnover and Market
  • 批准号:
  • 批准年份:
    2024
  • 资助金额:
    万元
  • 项目类别:
    外国学者研究基金
Exploring the Intrinsic Mechanisms of CEO Turnover and Market Reaction: An Explanation Based on Information Asymmetry
  • 批准号:
    W2433169
  • 批准年份:
    2024
  • 资助金额:
    万元
  • 项目类别:
    外国学者研究基金项目

相似海外基金

Postdoctoral Fellowship: STEMEdIPRF: Exploring the use of mechanistic reasoning in undergraduate physiology education
博士后奖学金:STEMEdIPRF:探索机械推理在本科生理学教育中的应用
  • 批准号:
    2327451
  • 财政年份:
    2024
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
Exploring the Impact of Future Land Use Change on Global Air Quality and Nutrient Deposition
探索未来土地利用变化对全球空气质量和养分沉积的影响
  • 批准号:
    2223070
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Standard Grant
A roadmap to a toolkit: exploring intervention opportunities for tobacco and cannabis co-use in the Further Education sector
工具包路线图:探索继续教育部门烟草和大麻共同使用的干预机会
  • 批准号:
    ES/Y008227/1
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Fellowship
Exploring affect-motivated alcohol use as a value-based decision-making process
探索情感驱动的饮酒作为基于价值的决策过程
  • 批准号:
    10738470
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
The Siyaphambili Substance Use Study: Exploring substance use and its treatment in the context of achieving sustained ART adherence among female sex workers
Siyaphambili 物质使用研究:在女性性工作者实现持续 ART 坚持的背景下探索物质使用及其治疗
  • 批准号:
    10619245
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
Portfolio of compositions: Creating electroacoustic works through the sonification of recursive neural networks, and exploring the creative use of in
作品组合:通过递归神经网络的发声创作电声作品,并探索in的创造性运用
  • 批准号:
    2886370
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Studentship
Looking through the bottle: Exploring alcohol use among emergency medical service providers
透过瓶子:探索紧急医疗服务提供者的酒精使用情况
  • 批准号:
    10712278
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
Exploring Long-Term Care Housing Solutions for Older Adults Experiencing Homelessness and Substance Use: An Innovative Model
探索针对无家可归和吸毒老年人的长期护理住房解决方案:创新模式
  • 批准号:
    484607
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Fellowship Programs
Crisis Intervention Teams in Corrections: Exploring the impact on officer use of force and the well-being of incarcerated people with mental illness
惩教中的危机干预小组:探讨对警官使用武力以及患有精神疾病的被监禁者福祉的影响
  • 批准号:
    10734687
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
Building digital replicas of unpaid carers to better meet community needs: : Exploring the use of Personal Digital Twin (PDT) technology
构建无薪护理人员的数字复制品,以更好地满足社区需求::探索个人数字孪生(PDT)技术的使用
  • 批准号:
    2880842
  • 财政年份:
    2023
  • 资助金额:
    $ 5.93万
  • 项目类别:
    Studentship
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了