SenSE: Artificial Intelligence-enabled Multimodal Stress Sensing for Precision Health

精准健康和精神病学的共同目标是精确测量相关症状，并以高精度维持心理健康。最佳心理健康做法提出，需要识别压力等风险因素的早期迹象。然而，心理健康研究主要侧重于通过自我报告数据（如调查）进行后期症状评估。因此，心理健康的临床实践主要包括反应性治疗。这笔赠款旨在通过开发一个支持人工智能（AI）的平台来持续精确地测量压力，从而促进精确的心理健康。该平台利用了来自类似皮肤的可穿戴设备的数据，该可穿戴设备测量皮质醇（来自汗液的压力激素），并且通过“重新利用”数十亿现有移动的和计算设备中可用的信号，利用了基于估计源自“战斗或逃跑”压力反应的肌肉硬度变化的传感技术的数据。这些数据流将使用机器学习（ML）算法进行组合，以优化数据收集，功耗和准确性。由于压力及其对心理健康恶化的影响是普遍存在的，这项工作的更广泛影响可能是巨大的，也是精神病学精确健康新研究的基础。该项目的总体目标是开发一个多模态传感平台，利用AI/ML算法优化应力预测和硬件性能。第一步是验证皮肤启发的可穿戴设备用于实验室压力测量，然后重新设计，并作为一个连续的野外设备进行验证。它具有一组生理传感器，用于收集心率变异性（HRV）和皮肤电活动（EDA）数据，这些信号与自主神经系统（ANS）反应直接相关。两种类型的传感器用于连续测量皮质醇水平将测试高精度和选择性。 然而，在野外连续处理和传输传感器数据需要大量的电池容量。为了解决这个问题，第二步将可穿戴设备与被动生物力学传感器和AI/ML算法相结合，以优化野外连续压力检测。另外，无源传感器将计算机外围数据（例如，小鼠、触控板、智能手机屏幕）转化为与“战斗或逃跑”应激反应相关的肌肉硬度相关的参数。例如，系统使用逆滤波技术来近似从鼠标位移导出的质量弹簧阻尼器（MSD）模型或从触控板上的手指下方的区域导出的力模型。 该系统采用几种AI/ML算法，包括a）压缩感知以优化能效，B）自动编码器模型以校正伪影、丢失数据或传感器故障，c）主动学习以发现压力事件和标签的最佳收集时间，以及d）云计算驱动的数据收集和处理以基于最佳可用数据进行预测。这项工作的智力优势包括：1）多模态压力监测可穿戴设备，可从汗液和其他生理信号中测量皮质醇; 2）生物力学传感算法，可将运动和触摸数据重新用于肌肉硬度; 3）AI/ML算法，可整合这些数据以优化可穿戴设备和智能手机的功耗，以高精度学习理想的传感场景，提高隐私，优化数据标签，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A shared objective of precision health and psychiatry is to precisely measure relevant symptoms and sustain mental health with high precision. Best mental health practices propose the need to identify the early signs of risk factors such as stress. However, mental health research focuses primarily on late-stage symptom assessment via self-report data such as surveys. As a result, clinical practice for mental health mostly consists of reactive treatments. This grant seeks to advance precision mental health by developing an Artificial Intelligence (AI) -enabled platform for continuous and precise measurements of stress. This platform leverages data from a skin-like wearable device that measures cortisol, a stress hormone from sweat, and from sensing techniques based on estimating muscle stiffness changes derived from “fight or flight” stress response, by “repurposing” signals available in billions of existing mobile and computing devices. These data streams will be combined using Machine Learning (ML) algorithms for optimizing data collection, power consumption, and accuracy. Since stress and its effect on mental health deterioration are pervasive, the broader impact of this work could be enormous as well as the basis for new research on precision health in psychiatry. The overall goal of this project is to develop a multimodal sensing platform leveraging AI/ML algorithms to optimize stress prediction and hardware performance. The first step involves validating the skin-inspired wearable for lab stress measurements, followed by redesigning, and validating as a continuous in-the-wild device. It features a set of physiological sensors for collecting heart rate variability (HRV) and electrodermal activity (EDA) data, signals directly correlated with the autonomous nervous system (ANS) response. Two types of sensors for continuous measurement of cortisol level will be tested for high accuracy and selectivity. However, processing and transmitting sensor data continuously in-the-wild requires significant battery capacity. To address this issue, the second step combines the wearable with passive biomechanical sensors and AI/ML algorithms to optimize for continuous stress detection in-the-wild. Additionally, passive sensors transform computer peripheral data (e.g., mice, trackpads, smartphone screens) into parameters correlated to muscle stiffness linked to the “fight or flight” stress response. For example, the system uses inverse filtering techniques to approximate mass-spring-damper (MSD) models derived from mouse displacements or force models derived from the area under the finger on a trackpad. The system employs several AI/ML algorithms including a) compressive sensing to optimize energy efficiency, b) autoencoder models to correct for artifacts, missing data or sensor failures, c) active learning to discover optimal collection times of stress events and labels, and d) cloud computing powered data collection and processing to make predictions based on the best available data. The intellectual merits of this work include 1) a multimodal stress monitoring wearable that measures cortisol from sweat and other physiological signals, 2) biomechanical sensing algorithms that repurpose movement and touch data into muscle stiffness, and 3) AI/ML algorithms that integrate this data to optimize wearable and smartphone power usage, learn ideal sensing scenarios with high precision, improve privacy, optimize data labeling, and optimize the early prediction of stress.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.