CAREER: Automatic Speech-Based Longitudinal Emotion and Mood Recognition for Mental Health Monitoring and Treatment

对精神健康障碍患者的有效治疗和监测是一项持久的社会挑战。定期监测增加了获得预防性治疗的机会，但考虑到对卫生保健提供者的高要求，费用往往令人望而却步或不可行。然而，这对患有双相情感障碍(BPD)的人来说是至关重要的，BPD是一种慢性精神疾病，其特征是情绪在健康和病理状态之间转换。向病态的转变与个人、社会、职业功能和情绪调节方面的严重干扰有关。这个学院早期职业发展计划(Career)项目通过利用言语、情绪和情绪之间的联系，研究基于言语的情绪监测的新方法。该方法包括处理具有短期变化(语音)的数据，估计中期变化(情绪)，然后使用情绪中的模式来识别长期变化(情绪)。教育推广活动包括一个设计挑战，它是由非营利性科学教育组织irisired创建的，旨在向服务不足的儿童及其父母传授非正式学习环境中的情感识别。这项研究探索了对自然的、纵向的语音数据进行建模并将情感模式与情绪相关联的方法，以应对语音情感识别和辅助技术中当前的挑战，这些挑战包括：概括性、健壮性和性能。这些方法概括为其症状包括非典型情绪的情况，如创伤后应激障碍、焦虑、抑郁和压力。这项研究将情绪作为简化言语和情绪之间映射的中间步骤；情绪失调是BPD的常见症状。情绪随着时间的推移被量化为价态和活跃度，以提高泛化能力。控制干扰调制以提高稳健性。它们共同导致了一系列低维的次要特征，这些特征的变化是由情绪引起的。这些次要特征被分割，以创建对情感的更粗略的时间描述。这提供了一种在语音(快速变化的信号)和用户状态(缓慢变化的信号)之间映射的方法，从而推进了最先进的技术。研究结果定量地揭示了情绪变化与用户状态变化之间的关系，为情绪识别和辅助技术领域的研究提供了新的方向和纽带。对使用时间序列技术对情感数据建模的关注导致了情感识别和辅助技术算法设计的突破。

Effective treatment and monitoring for individuals with mental health disorders is an enduring societal challenge. Regular monitoring increases access to preventative treatment, but is often cost prohibitive or infeasible given high demands placed on health care providers. Yet, it is critical for individuals with Bipolar Disorder (BPD), a chronic psychiatric illness characterized by mood transitions between healthy and pathological states. Transitions into pathological states are associated with profound disruptions in personal, social, vocational functioning, and emotion regulation. This Faculty Early Career Development Program (CAREER) project investigates new approaches in speech-based mood monitoring by taking advantage of the link between speech, emotion, and mood. The approach includes processing data with short-term variation (speech), estimating mid-term variation (emotion), and then using patterns in emotion to recognize long-term variation (mood). The educational outreach includes a design challenge, created with Iridescent, a science education nonprofit, that teaches emotion recognition to underserved children and their parents in informal learning settings. The research investigates methods to model naturalistic, longitudinal speech data and associate emotion patterns with mood, addressing current challenges in speech emotion recognition and assistive technology that include: generalizability, robustness, and performance. The approaches generalize to conditions whose symptoms include atypical emotion, such as post-traumatic stress disorder, anxiety, depression, and stress. The research forwards emotion as an intermediate step to simplify the mapping between speech and mood; emotion dysregulation is a common BPD symptom. Emotion is quantified over time in terms of valence and activation to improve generalizability. Nuisance modulations are controlled to improve robustness. Together, they result in a set of low-dimensional secondary features whose variations are due to emotion. These secondary features are segmented to create a coarser temporal description of emotion. This provides a means to map between speech (a quickly varying signal) and user state (a slowly varying signal), advancing the state-of-the-art. The results provide quantitative insight into the relationship between emotion variation and user state variation, providing new directions and links between the fields of emotion recognition and assistive technology. The focus on modeling emotional data using time series techniques results in breakthroughs in the design of emotion recognition and assistive technology algorithms.