权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RI: Small: Advancing the Science of Generalizable and Personalizable Speech-Centered Self-Report Emotion Classifiers

RI：小：推进以语音为中心的可概括和个性化的自我报告情绪分类器的科学

基本信息

批准号：
2230172
负责人：
Emily Provost
金额：
$ 60万
依托单位：
Regents of the University of Michigan - Ann Arbor
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-10-01 至 2025-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2230172&HistoricalAwards=false
关键词：
RI Small Advancing Science Generalizable

项目摘要

The goal of the project is to create new and personalized speech emotion recognition approaches and to use these approaches to investigate how changes in emotion are related to changes in mental health. The first step is accurately measuring how a person’s emotions vary over the course of a day, a week, a month, or even a year. However, the only approaches currently available to do so involve actively asking a user how they feel multiple times per day. Users are often willing to do this over shorter periods of time, but over longer periods of time this can be quite taxing. Fortunately, speech data are often easy to capture and conveys information about emotion. However, most approaches in speech emotion recognition are not focused on how the user feels and instead are focused on predicting how an outside group of people would label that user’s feeling. The goal of the project is to refocus automatic emotion classification on the user themselves. In the future, this will allow us to easily collect information about a user’s emotion leading to new investigations into how changes in emotions are associated with risk factors for changes in health.The goal of the presented research objectives is to advance the state-of-the-art in robust and generalizable personalized speech (acoustics + language) self-report emotion recognition classifiers and to investigate how measures created using these classifiers will allow researchers to intuit changes in mental health symptom severity in a clinical population of individuals at risk for suicidality. The field of automatic speech emotion recognition is almost exclusively focused on estimating how an outside group of observers would perceive a given emotional display (i.e., perception-of-other). Yet, when the focus is on the ultimate use cases of this technology, e.g., mental health symptom severity tracking, this is often not what is needed. Instead, symptom severity tracking often needs information about how a given individual is interpreting their own emotional experiences (i.e., self-report). For example, changing patterns in self-report are associated with changes in depression severity. Yet, these changes are currently measurable only through active participation, in which individuals are regularly asked to describe their emotional experiences using self-report measures (e.g., Ecological Momentary Assessment, EMA) longitudinally, multiple times per day, which can be quite expensive both in terms of cost and participant burden. The project team envisions a future in which audio can be passively collected and used to automatically infer self-reported emotion, but there has been limited attention to the design of such classifiers due to persistent challenges associated with accurately estimating self-reported emotion, including cognitive bias, context, and the difference between self-report and emotional experiences. The project team will accomplish these goals by: 1) creating classifiers that are robust and generalizable using new metrics that encourage models to attend to the same acoustic and language cues as human observers; 2) personalizing classifiers to users longitudinally, and 3) evaluating the effectiveness of self-report emotion classifiers by predicting changes in mental health symptom severity using an existing real-world dataset annotated with mental health symptom severity (risk of suicide). The presented approaches will forward investigations into how to use passively collected audio data to estimate changes in risk factors for health changes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目标是创建新的和个性化的语音情感识别方法，并使用这些方法来研究情感的变化如何与心理健康的变化相关。第一步是准确测量一个人的情绪在一天、一周、一个月甚至一年中的变化。然而，目前唯一可用的方法是每天多次主动询问用户感觉如何。用户通常愿意在较短的时间内这样做，但在较长的时间内，这可能是相当繁重的。幸运的是，语音数据通常很容易捕获并传达有关情感的信息。然而，语音情感识别中的大多数方法并不关注用户的感受，而是专注于预测外部人群将如何标记用户的感受。该项目的目标是将自动情感分类重新聚焦于用户本身。在未来，这将使我们能够很容易地收集有关用户情绪的信息，从而对情绪变化如何与健康变化的风险因素相关联进行新的调查。所提出的研究目标的目的是在鲁棒的和可推广的个性化语音方面推进最新技术水平（acoustics + language）（声学+语言）报告情绪识别分类器，并研究如何使用这些分类器创建的措施将使研究人员直观地了解心理健康症状严重程度的变化，有自杀倾向的临床人群。自动语音情感识别的领域几乎完全集中在估计外部观察者组将如何感知给定的情感显示（即，感知他人）。然而，当重点放在这项技术的最终用例时，例如，心理健康症状严重程度跟踪，这往往是不需要的。相反，症状严重性跟踪通常需要关于给定个体如何解释他们自己的情绪体验的信息（即，自我报告）。例如，自我报告模式的变化与抑郁症严重程度的变化有关。然而，这些变化目前只能通过积极参与来衡量，其中个人定期被要求使用自我报告措施来描述他们的情感体验（例如，生态瞬时评估，EMA）纵向，每天多次，这可能是相当昂贵的成本和参与者的负担。项目团队设想了一个未来，其中音频可以被动收集并用于自动推断自我报告的情绪，但由于与准确估计自我报告的情绪相关的持续挑战，包括认知偏见，上下文以及自我报告和情绪体验之间的差异，对此类分类器的设计关注有限。项目团队将通过以下方式实现这些目标：1）使用新的度量标准创建健壮且可推广的分类器，这些度量标准鼓励模型与人类观察者一样关注相同的声学和语言线索; 2）对用户纵向个性化分类器，以及3）通过使用现有的真实的-用精神健康症状严重程度（自杀风险）注释的世界数据集。所提出的方法将推动调查如何使用被动收集的音频数据来估计健康变化的风险因素的变化。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。