Collaborative Research: SCH: Personalized Watch-based Fall Risk Analysis and Detection with Cross Modal Learning

跌倒是老年人发病和死亡的一个重要原因。一种可靠且低成本的跌倒风险评估和跌倒检测解决方案将使老年人能够独立生活，并减少因跌倒而导致的医疗费用。可穿戴设备已被开发用于检测“硬跌倒”，即导致伤害的跌倒。然而，许多老年人跌倒并不会造成身体伤害（“软性跌倒”）。这些与体重转移活动有关，如转身和坐立转换。事实上，控制躯干（“核心”）的位置和运动的能力对于在体重转移过程中协调四肢的运动是必不可少的。该项目的目标是将个人真实的肢体核心动态与通过商品腕表和另一侧臀部的手机收集的加速度计数据结合起来，以提高对硬和软跌倒的检测。通过使用最先进的机器学习算法实时学习肢体核心动态，将为每个用户创建个性化的跌倒风险分析和检测模型。我们还将评估老年患者使用这项技术的看法和偏好，并评估他们对持续收集和共享健康数据以改善健康的态度。软件系统、真实世界的步态和体重转移运动以及相关的加速度计数据将免费提供给任何对机器学习研究健康状况以及跌倒风险和分析感兴趣的机构、研究者或研究生。该项目将培训研究生和本科生的技术技能（机器学习、可穿戴技术和数据分析技能）以及与长期护理机构的老年人一起工作的人际交往技能。虽然已经开发了许多结合人工智能（AI）和机器学习算法的跌倒检测设备，但该项目侧重于个性化跌倒风险检测。该项目将探索使用与体重转移相关的老年人运动的运动学测量，以使多任务和多模式机器学习算法能够个性化跌倒风险检测。开发了一种基于小样本的深度学习算法，该算法利用多任务和多模态学习框架优化了个体运动学特征。其次，团队将分析Azure Kinect系统捕获的运动转换，以识别加速度计数据与完整骨骼框架之间的关系，重点是肢体核心动力学。具体来说，我们的目标是确定生成对抗网络（GANs）是否可以用于从少量身体运动数据、智能手表和手机加速数据中增强缺失模态，这些数据直接收集自最有可能摔倒的老年参与者，即生活在辅助生活中心的老年人。最后，我们将评估老年参与者对持续使用可穿戴设备进行跌倒风险分析和检测的感知和态度。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long-term care facilities. While numerous fall detection devices incorporating artificial intelligence (AI) and machine learning algorithms have been developed, this project focuses on personalizing fall risk detection. This project will explore the use of kinematic measurements of an elderly individual’s movements associated with weight transfer to enable multi-task and multi-modal machine learning algorithms to personalize fall risk detection. A small-sample-based deep learning algorithm optimized to incorporate individual kinematic characteristics using multi-task and multi-modal learning frameworks is developed. Second, the team will analyze the movement transitions captured by the Azure Kinect system in order to identify relationships between the accelerometer data and the complete skeletal frame with an emphasis on the limb-core dynamics. Specifically, our goal is to determine whether or not Generative Adversarial Networks (GANs) can be used to augment missing modality from a small amount of body motion data, smartwatch and phone acceleration data collected directly from elderly participants who are at most risk of falling, namely those living in an assisted living center. Finally, we will evaluate the perception and attitudes of the elderly participants towards the continuous use of wearable devices for fall risk analysis and detection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.