SenSE: Artificial Intelligence-enabled Multimodal Stress Sensing for Precision Health

精确健康和精神病学的共同目标是精确测量相关症状，以高精度维持精神健康。最佳心理健康实践建议需要识别诸如压力等风险因素的早期迹象。然而，心理健康研究主要侧重于通过自我报告数据（如调查）来评估晚期症状。因此，心理健康的临床实践主要由反应性治疗组成。该拨款旨在通过开发一个人工智能（AI）支持的平台，用于连续和精确测量压力，从而推进精确的心理健康。该平台利用来自皮肤状可穿戴设备的数据，该设备测量汗液中的应激激素皮质醇，以及基于估计“战斗或逃跑”应激反应引起的肌肉僵硬变化的传感技术，通过“重新利用”数十亿现有移动和计算设备中可用的信号。这些数据流将使用机器学习（ML）算法进行组合，以优化数据收集、功耗和准确性。由于压力及其对精神健康恶化的影响是普遍存在的，这项工作的广泛影响可能是巨大的，同时也是精神病学精确健康新研究的基础。该项目的总体目标是开发一个利用AI/ML算法优化应力预测和硬件性能的多模态传感平台。第一步是验证皮肤启发的可穿戴设备在实验室的应力测量，然后是重新设计，并验证作为一个连续的野外设备。它具有一组生理传感器，用于收集心率变异性（HRV）和皮电活动（EDA）数据，这些信号与自主神经系统（ANS）反应直接相关。两种类型的传感器连续测量皮质醇水平将测试高准确性和选择性。然而，在野外连续处理和传输传感器数据需要很大的电池容量。为了解决这个问题，第二步将可穿戴设备与被动生物力学传感器和AI/ML算法相结合，以优化野外连续应力检测。此外，被动传感器将计算机外围数据（例如，鼠标、触控板、智能手机屏幕）转换为与“战斗或逃跑”应激反应相关的肌肉僵硬度相关的参数。例如，该系统使用反滤波技术来近似从鼠标位移导出的质量-弹簧-阻尼器（MSD）模型，或从触控板上手指下方的面积导出的力模型。该系统采用了几种AI/ML算法，包括a)压缩感知优化能源效率，b)自动编码器模型纠正工件，缺失数据或传感器故障，c)主动学习以发现压力事件和标签的最佳收集时间，d)云计算支持的数据收集和处理，基于最佳可用数据进行预测。这项工作的智力优势包括：1)测量汗液和其他生理信号中的皮质醇的多模态应力监测可穿戴设备；2)将运动和触摸数据重新用于肌肉僵硬的生物力学传感算法；3)整合这些数据以优化可穿戴设备和智能手机的功耗，高精度学习理想的传感场景，改善隐私，优化数据标记，优化压力的早期预测的AI/ML算法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A shared objective of precision health and psychiatry is to precisely measure relevant symptoms and sustain mental health with high precision. Best mental health practices propose the need to identify the early signs of risk factors such as stress. However, mental health research focuses primarily on late-stage symptom assessment via self-report data such as surveys. As a result, clinical practice for mental health mostly consists of reactive treatments. This grant seeks to advance precision mental health by developing an Artificial Intelligence (AI) -enabled platform for continuous and precise measurements of stress. This platform leverages data from a skin-like wearable device that measures cortisol, a stress hormone from sweat, and from sensing techniques based on estimating muscle stiffness changes derived from “fight or flight” stress response, by “repurposing” signals available in billions of existing mobile and computing devices. These data streams will be combined using Machine Learning (ML) algorithms for optimizing data collection, power consumption, and accuracy. Since stress and its effect on mental health deterioration are pervasive, the broader impact of this work could be enormous as well as the basis for new research on precision health in psychiatry. The overall goal of this project is to develop a multimodal sensing platform leveraging AI/ML algorithms to optimize stress prediction and hardware performance. The first step involves validating the skin-inspired wearable for lab stress measurements, followed by redesigning, and validating as a continuous in-the-wild device. It features a set of physiological sensors for collecting heart rate variability (HRV) and electrodermal activity (EDA) data, signals directly correlated with the autonomous nervous system (ANS) response. Two types of sensors for continuous measurement of cortisol level will be tested for high accuracy and selectivity. However, processing and transmitting sensor data continuously in-the-wild requires significant battery capacity. To address this issue, the second step combines the wearable with passive biomechanical sensors and AI/ML algorithms to optimize for continuous stress detection in-the-wild. Additionally, passive sensors transform computer peripheral data (e.g., mice, trackpads, smartphone screens) into parameters correlated to muscle stiffness linked to the “fight or flight” stress response. For example, the system uses inverse filtering techniques to approximate mass-spring-damper (MSD) models derived from mouse displacements or force models derived from the area under the finger on a trackpad. The system employs several AI/ML algorithms including a) compressive sensing to optimize energy efficiency, b) autoencoder models to correct for artifacts, missing data or sensor failures, c) active learning to discover optimal collection times of stress events and labels, and d) cloud computing powered data collection and processing to make predictions based on the best available data. The intellectual merits of this work include 1) a multimodal stress monitoring wearable that measures cortisol from sweat and other physiological signals, 2) biomechanical sensing algorithms that repurpose movement and touch data into muscle stiffness, and 3) AI/ML algorithms that integrate this data to optimize wearable and smartphone power usage, learn ideal sensing scenarios with high precision, improve privacy, optimize data labeling, and optimize the early prediction of stress.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.