CAREER: Mobile Sensor-Based Adaptive Emotion Prediction and Feedback Delivery

职业：基于移动传感器的自适应情绪预测和反馈传递

• 批准号: 2047296
• 负责人: Akane Sano
• 金额: $ 55万
• 依托单位: William Marsh Rice University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-04-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047296&HistoricalAwards=false
• 关键词: CAREER; Mobile; Sensor; Based; Adaptive

Recent advances in wearable, mobile technologies and the Internet of Things enable us to collect moment-to-moment physiological, behavioral, social, and environmental data on mobility pattens, sleep, central and peripheral nervous system activity, and social interactions, all without disrupting daily routines. Such data show a potential for revolutionizing how we diagnose, ameliorate, and prevent health disorders. This project designs, implements, and evaluates personalized, adaptive algorithms to detect and predict emotional states using multimodal sensor data, and to provide feedback to users to improve management of mental state. The emotion detection and feedback algorithms adapt to changing human physiology, behavior, context, and preferences. This research will result in emotion assistive technologies that enhance human performance, health, and wellbeing, thus improving quality of life. The project will provide a platform for integrating mobile sensor data and providing feedback to subjects valuable to a broad range of populations for personalized medicine. The project will yield insights into ethical issues in the development, evaluation, and use of human data and artificial intelligence technology. The research activities will train graduate students, be integrated into classroom curricula in data science, and provide research opportunitites and summer internships for undergraduate and high school students. This project will develop dynamic emotion modeling and feedback systems to harness multimodal human data for automatic human emotion recognition and prediction. The system will provide safe and personalized feedback delivery to help manage emotional states. This goal will be achieved by addressing three fundamental research challenges: (1) multi-modal, multi-timescale physiological and behavioral pattern interpretation and analysis; (2) adaptive emotion label sampling for effective emotion detection and prediction; and (3) safe and sustainable automatic personalized feedback delivery. The resulting emotion detection and feedback systems will be integrated to help users in poor health manage emotion. The performance, usability, safety, and effectiveness of the technologies will be evaluated via human subject studies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

可穿戴、移动的技术和物联网的最新进展使我们能够收集有关移动模式、睡眠、中枢和外周神经系统活动以及社交互动的实时生理、行为、社交和环境数据，而无需中断日常生活。 这些数据显示了我们诊断、改善和预防健康疾病的方式发生革命性变化的潜力。该项目设计，实施和评估个性化的自适应算法，以使用多模态传感器数据检测和预测情绪状态，并向用户提供反馈，以改善精神状态的管理。 情感检测和反馈算法适应不断变化的人类生理、行为、上下文和偏好。这项研究将导致情感辅助技术，提高人类的表现，健康和福祉，从而提高生活质量。该项目将提供一个平台，用于整合移动的传感器数据，并向对广泛人群有价值的受试者提供反馈，以进行个性化医疗。该项目将深入了解人类数据和人工智能技术的开发、评估和使用中的伦理问题。 研究活动将培训研究生，融入数据科学的课堂课程，并为本科生和高中生提供研究机会和暑期实习机会。该项目将开发动态情感建模和反馈系统，以利用多模态人类数据进行自动人类情感识别和预测。 该系统将提供安全和个性化的反馈，以帮助管理情绪状态。这一目标将通过解决三个基础研究挑战来实现：（1）多模态，多时间尺度的生理和行为模式解释和分析;（2）自适应情绪标签采样，用于有效的情绪检测和预测;（3）安全和可持续的自动个性化反馈交付。由此产生的情绪检测和反馈系统将被集成，以帮助健康状况不佳的用户管理情绪。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Semi-Supervised Learning for Wearable-based Momentary Stress Detection in the Wild

• DOI: 10.1145/3596246
• 发表时间: 2023-06
• 期刊: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
• 作者: Han Yu;Akane Sano

Toward the Analysis of Office Workers’ Mental Indicators Based on Wearable, Work Activity, and Weather Data

基于可穿戴设备、工作活动和天气数据的办公室职员心理指标分析

• 发表时间: 2021
• 期刊: International Conference on Activity and Behavior Computing (ABC
• 作者: Nishimura, Yusuke;Hossain, Tahera;Sano, Akane;Isomura, Shota;Arakawa, Yutaka;Inoue, Sozo
• 通讯作者: Inoue, Sozo

Modality Fusion Network and Personalized Attention in Momentary Stress Detection in the Wild

• DOI: 10.1109/acii52823.2021.9597459
• 发表时间: 2021-07
• 期刊: 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII)
• 作者: Han Yu;T. Vaessen;I. Myin‐Germeys;Akane Sano

Health Label and Behavioral Feature Prediction Using Bayesian Hierarchical Vector Autoregression Models

使用贝叶斯分层向量自回归模型进行健康标签和行为特征预测

• DOI: 10.1109/embc46164.2021.9630732
• 发表时间: 2021
• 期刊: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC
• 作者: Lyon, Ethan N.;Victor, Luis H.;Sano, Akane
• 通讯作者: Sano, Akane

Bias Reducing Multitask Learning on Mental Health Prediction

• DOI: 10.1109/acii55700.2022.9953850
• 发表时间: 2022-08
• 期刊: 2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)
• 作者: Khadija Zanna;K. Sridhar;Han Yu;Akane Sano