权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Automatic Speech-Based Longitudinal Emotion and Mood Recognition for Mental Health Monitoring and Treatment

职业：基于语音的自动纵向情感和情绪识别，用于心理健康监测和治疗

基本信息

批准号：
1651740
负责人：
Emily Provost
金额：
$ 54.88万
依托单位：
Regents of the University of Michigan - Ann Arbor
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-02-01 至 2023-01-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1651740&HistoricalAwards=false
关键词：
CAREER Automatic Speech Based Longitudinal

项目摘要

Effective treatment and monitoring for individuals with mental health disorders is an enduring societal challenge. Regular monitoring increases access to preventative treatment, but is often cost prohibitive or infeasible given high demands placed on health care providers. Yet, it is critical for individuals with Bipolar Disorder (BPD), a chronic psychiatric illness characterized by mood transitions between healthy and pathological states. Transitions into pathological states are associated with profound disruptions in personal, social, vocational functioning, and emotion regulation. This Faculty Early Career Development Program (CAREER) project investigates new approaches in speech-based mood monitoring by taking advantage of the link between speech, emotion, and mood. The approach includes processing data with short-term variation (speech), estimating mid-term variation (emotion), and then using patterns in emotion to recognize long-term variation (mood). The educational outreach includes a design challenge, created with Iridescent, a science education nonprofit, that teaches emotion recognition to underserved children and their parents in informal learning settings. The research investigates methods to model naturalistic, longitudinal speech data and associate emotion patterns with mood, addressing current challenges in speech emotion recognition and assistive technology that include: generalizability, robustness, and performance. The approaches generalize to conditions whose symptoms include atypical emotion, such as post-traumatic stress disorder, anxiety, depression, and stress. The research forwards emotion as an intermediate step to simplify the mapping between speech and mood; emotion dysregulation is a common BPD symptom. Emotion is quantified over time in terms of valence and activation to improve generalizability. Nuisance modulations are controlled to improve robustness. Together, they result in a set of low-dimensional secondary features whose variations are due to emotion. These secondary features are segmented to create a coarser temporal description of emotion. This provides a means to map between speech (a quickly varying signal) and user state (a slowly varying signal), advancing the state-of-the-art. The results provide quantitative insight into the relationship between emotion variation and user state variation, providing new directions and links between the fields of emotion recognition and assistive technology. The focus on modeling emotional data using time series techniques results in breakthroughs in the design of emotion recognition and assistive technology algorithms.

对精神健康障碍患者进行有效的治疗和监测是一项持久的社会挑战。定期监测增加了获得预防性治疗的机会，但由于对卫生保健提供者的要求很高，因此往往成本过高或不可行。然而，这对双相情感障碍（BPD）患者至关重要，BPD是一种慢性精神疾病，其特征是健康和病理状态之间的情绪转变。向病理状态的转变与个人、社会、职业功能和情绪调节的深刻破坏有关。这个教师早期职业发展计划（CAREER）项目通过利用语音，情绪和情绪之间的联系，研究了基于语音的情绪监测的新方法。该方法包括处理具有短期变化（语音）的数据，估计中期变化（情绪），然后使用情绪中的模式来识别长期变化（情绪）。教育推广活动包括一项设计挑战，由科学教育非营利组织Iridescent创建，在非正式学习环境中向服务不足的儿童及其父母教授情感识别。该研究探讨了建模自然，纵向语音数据和情感模式与情绪相关联的方法，解决了当前语音情感识别和辅助技术的挑战，包括：概括性，鲁棒性和性能。这些方法适用于症状包括非典型情绪的情况，如创伤后应激障碍、焦虑、抑郁和压力。该研究将情绪作为简化言语和情绪之间映射的中间步骤;情绪失调是BPD的常见症状。随着时间的推移，情绪被量化的效价和激活，以提高概括性。控制营养调节以提高鲁棒性。它们共同导致了一组低维的次要特征，其变化是由于情感。这些次要特征被分段以创建情感的更粗略的时间描述。这提供了一种方法来映射语音（一个快速变化的信号）和用户状态（一个缓慢变化的信号），推进国家的最先进的。结果提供定量洞察情感变化和用户状态变化之间的关系，提供新的方向和情感识别和辅助技术领域之间的联系。对使用时间序列技术建模情感数据的关注导致了情感识别和辅助技术算法设计的突破。

项目成果

期刊论文数量（17）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Read speech voice quality and disfluency in individuals with recent suicidal ideation or suicide attempt

阅读最近有自杀意念或自杀企图的个人的语音质量和不流畅性

DOI：
10.1016/j.specom.2021.05.004
发表时间：
2021
期刊：
Speech Communication
影响因子：
3.2
作者：
Stasak, Brian;Epps, Julien;Schatten, Heather T.;Miller, Ivan W.;Provost, Emily Mower;Armey, Michael F.
通讯作者：
Armey, Michael F.

Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation

DOI：
10.1609/aaai.v33i01.33015581
发表时间：
2019-07
期刊：
ArXiv
影响因子：
0
作者：
Yonghao Xu;Bo Du;Lefei Zhang;Qian Zhang;Guoli Wang;Liangpei Zhang
通讯作者：
Yonghao Xu;Bo Du;Lefei Zhang;Qian Zhang;Guoli Wang;Liangpei Zhang

Predicting the distribution of emotion perception: capturing inter-rater variability

DOI：
10.1145/3136755.3136792
发表时间：
2017-11
期刊：
Proceedings of the 19th ACM International Conference on Multimodal Interaction
影响因子：
0
作者：
Biqiao Zhang;Georg Essl;E. Provost
通讯作者：
Biqiao Zhang;Georg Essl;E. Provost

MuSE: a Multimodal Dataset of Stressed Emotion

DOI：
发表时间：
2020-05
期刊：
影响因子：
0
作者：
Mimansa Jaiswal;Cristian-Paul Bara;Y. Luo;Mihai Burzo;Rada Mihalcea;E. Provost
通讯作者：
Mimansa Jaiswal;Cristian-Paul Bara;Y. Luo;Mihai Burzo;Rada Mihalcea;E. Provost

Towards Noise Robust Speech Emotion Recognition Using Dynamic Layer Customization

DOI：
10.1109/acii52823.2021.9597437
发表时间：
2021-09
期刊：
2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII)
影响因子：
0
作者：
Alex Wilf;E. Provost
通讯作者：
Alex Wilf;E. Provost