SenSE: Artificial Intelligence-enabled Multimodal Stress Sensing for Precision Health

精确健康和精神病学的一个共同目标是精确测量相关症状，并以高精度维持心理健康。最佳的心理健康实践建议，需要确定压力等风险因素的早期迹象。然而，心理健康研究主要集中在通过调查等自我报告数据进行晚期症状评估。因此，心理健康的临床实践主要包括反应性治疗。这笔赠款旨在通过开发一个支持人工智能(AI)的平台来持续和精确地测量压力，以促进精确的心理健康。该平台利用了一种皮肤状可穿戴设备的数据，该设备可以测量皮质醇，皮质醇是一种来自汗液的压力荷尔蒙；该平台还利用了传感技术的数据，这些技术基于对现有移动和计算设备中的数十亿个可用的信号进行“重新调整用途”，来估计“战斗或逃跑”压力反应产生的肌肉僵硬变化。这些数据流将使用机器学习(ML)算法进行组合，以优化数据收集、功耗和准确性。由于压力及其对精神健康恶化的影响是普遍存在的，这项工作的广泛影响可能是巨大的，也是精神病学中关于精确健康的新研究的基础。该项目的总体目标是开发一个多模式传感平台，利用AI/ML算法来优化应力预测和硬件性能。第一步是验证这款受皮肤启发的可穿戴设备是否可用于实验室压力测量，然后重新设计，并验证其是否为连续的野外设备。它具有一套生理传感器，用于收集心率变异性(HRV)和皮肤电活动(EDA)数据，这些信号与自主神经系统(ANS)反应直接相关。用于连续测量皮质醇水平的两种传感器将进行高精度和选择性测试。然而，在野外连续处理和传输传感器数据需要相当大的电池容量。为了解决这个问题，第二步将可穿戴设备与被动生物力学传感器和AI/ML算法相结合，以优化野外连续应力检测。此外，被动传感器将计算机外设数据(如鼠标、触摸板、智能手机屏幕)转换为与肌肉僵硬相关的参数，与“战斗或逃跑”的压力反应有关。例如，该系统使用逆滤波技术来近似质量-弹簧-阻尼器(MSD)模型，该模型来自鼠标位移，或者力模型，该模型来自触控板上手指下方的区域。该系统采用了几种AI/ML算法，包括a)压缩传感以优化能源效率，b)自动编码器模型以校正伪影、丢失数据或传感器故障，c)主动学习以发现压力事件和标签的最佳收集时间，以及d)云计算支持的数据收集和处理以基于最佳可用数据进行预测。这项工作的智力价值包括1)多模式压力监测可穿戴设备，测量汗液和其他生理信号中的皮质醇；2)生物力学传感算法，将运动和触摸数据重新调整为肌肉僵硬；3)AI/ML算法，整合这些数据以优化可穿戴设备和智能手机的电力使用，高精度地学习理想的传感场景，改善隐私，优化数据标签，并优化压力的早期预测。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A shared objective of precision health and psychiatry is to precisely measure relevant symptoms and sustain mental health with high precision. Best mental health practices propose the need to identify the early signs of risk factors such as stress. However, mental health research focuses primarily on late-stage symptom assessment via self-report data such as surveys. As a result, clinical practice for mental health mostly consists of reactive treatments. This grant seeks to advance precision mental health by developing an Artificial Intelligence (AI) -enabled platform for continuous and precise measurements of stress. This platform leverages data from a skin-like wearable device that measures cortisol, a stress hormone from sweat, and from sensing techniques based on estimating muscle stiffness changes derived from “fight or flight” stress response, by “repurposing” signals available in billions of existing mobile and computing devices. These data streams will be combined using Machine Learning (ML) algorithms for optimizing data collection, power consumption, and accuracy. Since stress and its effect on mental health deterioration are pervasive, the broader impact of this work could be enormous as well as the basis for new research on precision health in psychiatry. The overall goal of this project is to develop a multimodal sensing platform leveraging AI/ML algorithms to optimize stress prediction and hardware performance. The first step involves validating the skin-inspired wearable for lab stress measurements, followed by redesigning, and validating as a continuous in-the-wild device. It features a set of physiological sensors for collecting heart rate variability (HRV) and electrodermal activity (EDA) data, signals directly correlated with the autonomous nervous system (ANS) response. Two types of sensors for continuous measurement of cortisol level will be tested for high accuracy and selectivity. However, processing and transmitting sensor data continuously in-the-wild requires significant battery capacity. To address this issue, the second step combines the wearable with passive biomechanical sensors and AI/ML algorithms to optimize for continuous stress detection in-the-wild. Additionally, passive sensors transform computer peripheral data (e.g., mice, trackpads, smartphone screens) into parameters correlated to muscle stiffness linked to the “fight or flight” stress response. For example, the system uses inverse filtering techniques to approximate mass-spring-damper (MSD) models derived from mouse displacements or force models derived from the area under the finger on a trackpad. The system employs several AI/ML algorithms including a) compressive sensing to optimize energy efficiency, b) autoencoder models to correct for artifacts, missing data or sensor failures, c) active learning to discover optimal collection times of stress events and labels, and d) cloud computing powered data collection and processing to make predictions based on the best available data. The intellectual merits of this work include 1) a multimodal stress monitoring wearable that measures cortisol from sweat and other physiological signals, 2) biomechanical sensing algorithms that repurpose movement and touch data into muscle stiffness, and 3) AI/ML algorithms that integrate this data to optimize wearable and smartphone power usage, learn ideal sensing scenarios with high precision, improve privacy, optimize data labeling, and optimize the early prediction of stress.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.