权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RI: Small: Advancing the Science of Generalizable and Personalizable Speech-Centered Self-Report Emotion Classifiers

RI：小：推进以语音为中心的可概括和个性化的自我报告情绪分类器的科学

基本信息

批准号：
2230172
负责人：
Emily Provost
金额：
$ 60万
依托单位：
Regents of the University of Michigan - Ann Arbor
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-10-01 至 2025-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2230172&HistoricalAwards=false
关键词：
RI Small Advancing Science Generalizable

项目摘要

The goal of the project is to create new and personalized speech emotion recognition approaches and to use these approaches to investigate how changes in emotion are related to changes in mental health. The first step is accurately measuring how a person’s emotions vary over the course of a day, a week, a month, or even a year. However, the only approaches currently available to do so involve actively asking a user how they feel multiple times per day. Users are often willing to do this over shorter periods of time, but over longer periods of time this can be quite taxing. Fortunately, speech data are often easy to capture and conveys information about emotion. However, most approaches in speech emotion recognition are not focused on how the user feels and instead are focused on predicting how an outside group of people would label that user’s feeling. The goal of the project is to refocus automatic emotion classification on the user themselves. In the future, this will allow us to easily collect information about a user’s emotion leading to new investigations into how changes in emotions are associated with risk factors for changes in health.The goal of the presented research objectives is to advance the state-of-the-art in robust and generalizable personalized speech (acoustics + language) self-report emotion recognition classifiers and to investigate how measures created using these classifiers will allow researchers to intuit changes in mental health symptom severity in a clinical population of individuals at risk for suicidality. The field of automatic speech emotion recognition is almost exclusively focused on estimating how an outside group of observers would perceive a given emotional display (i.e., perception-of-other). Yet, when the focus is on the ultimate use cases of this technology, e.g., mental health symptom severity tracking, this is often not what is needed. Instead, symptom severity tracking often needs information about how a given individual is interpreting their own emotional experiences (i.e., self-report). For example, changing patterns in self-report are associated with changes in depression severity. Yet, these changes are currently measurable only through active participation, in which individuals are regularly asked to describe their emotional experiences using self-report measures (e.g., Ecological Momentary Assessment, EMA) longitudinally, multiple times per day, which can be quite expensive both in terms of cost and participant burden. The project team envisions a future in which audio can be passively collected and used to automatically infer self-reported emotion, but there has been limited attention to the design of such classifiers due to persistent challenges associated with accurately estimating self-reported emotion, including cognitive bias, context, and the difference between self-report and emotional experiences. The project team will accomplish these goals by: 1) creating classifiers that are robust and generalizable using new metrics that encourage models to attend to the same acoustic and language cues as human observers; 2) personalizing classifiers to users longitudinally, and 3) evaluating the effectiveness of self-report emotion classifiers by predicting changes in mental health symptom severity using an existing real-world dataset annotated with mental health symptom severity (risk of suicide). The presented approaches will forward investigations into how to use passively collected audio data to estimate changes in risk factors for health changes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目标是创建新的个性化语音情绪识别方法，并使用这些方法来研究情绪变化与心理健康变化之间的关系。第一步是准确测量一个人的情绪在一天、一周、一个月甚至一年内的变化。然而，目前唯一可用的方法是每天多次主动询问用户的感受。用户通常愿意在较短的时间内执行此操作，但在较长的时间内，这可能会非常费力。幸运的是，语音数据通常很容易捕获并传达有关情感的信息。然而，语音情感识别中的大多数方法并不关注用户的感受，而是专注于预测外部人群将如何标记用户的感受。该项目的目标是将自动情绪分类重新聚焦于用户本身。将来，这将使我们能够轻松收集有关用户情绪的信息，从而对情绪变化如何与健康变化的风险因素相关联进行新的调查。所提出的研究目标是推进最先进的稳健且可概括的个性化语音（声学+语言）自我报告情绪识别分类器，并研究使用这些分类器创建的测量如何使研究人员能够直观地了解临床人群心理健康症状严重程度的变化。有自杀风险的人。自动语音情感识别领域几乎完全专注于估计外部观察者群体如何感知给定的情感表现（即对他人的感知）。然而，当重点关注该技术的最终用例时，例如心理健康症状严重程度跟踪，这通常不是所需要的。相反，症状严重程度跟踪通常需要有关特定个体如何解释自己的情绪体验的信息（即自我报告）。例如，自我报告模式的变化与抑郁严重程度的变化有关。然而，这些变化目前只能通过积极参与来衡量，其中定期要求个人使用自我报告措施（例如生态瞬时评估，EMA）纵向描述他们的情感体验，每天多次，这在成本和参与者负担方面都可能相当昂贵。项目团队设想未来可以被动地收集音频并用于自动推断自我报告的情绪，但由于与准确估计自我报告的情绪相关的持续挑战（包括认知偏差、背景以及自我报告和情绪体验之间的差异），人们对此类分类器的设计的关注有限。项目团队将通过以下方式实现这些目标：1）使用新指标创建强大且可概括的分类器，鼓励模型关注与人类观察者相同的声音和语言线索； 2）纵向为用户个性化分类器，3）通过使用标注有心理健康症状严重程度（自杀风险）的现有现实数据集预测心理健康症状严重程度的变化来评估自我报告情绪分类器的有效性。所提出的方法将深入研究如何使用被动收集的音频数据来估计健康变化的风险因素的变化。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。