Collaborative Research: SCH:Personalized Watch-based Fall Risk Analysis and Detection with Cross Modal Learning

合作研究：SCH：通过跨模态学习进行基于手表的个性化跌倒风险分析和检测

• 批准号: 2123749
• 负责人: Anne Ngu
• 金额: $ 91.02万
• 依托单位: Texas State University - San Marcos
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2123749&HistoricalAwards=false
• 关键词: Collaborative; Research; SCH; Personalized; Watch

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long-term care facilities. While numerous fall detection devices incorporating artificial intelligence (AI) and machine learning algorithms have been developed, this project focuses on personalizing fall risk detection. This project will explore the use of kinematic measurements of an elderly individual’s movements associated with weight transfer to enable multi-task and multi-modal machine learning algorithms to personalize fall risk detection. A small-sample-based deep learning algorithm optimized to incorporate individual kinematic characteristics using multi-task and multi-modal learning frameworks is developed. Second, the team will analyze the movement transitions captured by the Azure Kinect system in order to identify relationships between the accelerometer data and the complete skeletal frame with an emphasis on the limb-core dynamics. Specifically, our goal is to determine whether or not Generative Adversarial Networks (GANs) can be used to augment missing modality from a small amount of body motion data, smartwatch and phone acceleration data collected directly from elderly participants who are at most risk of falling, namely those living in an assisted living center. Finally, we will evaluate the perception and attitudes of the elderly participants towards the continuous use of wearable devices for fall risk analysis and detection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

福尔斯是老年人发病和死亡的重要原因。 一种用于估计跌倒风险和检测福尔斯的稳健且低成本的解决方案将使老年人能够独立生活，并降低由于福尔斯而导致的医疗成本。 已经开发了可穿戴设备来检测“硬福尔斯”，即导致伤害的福尔斯。 然而，许多老年人的福尔斯跌倒并不造成身体伤害（“软福尔斯”）。 这些发生与体重转移活动，如转身和坐-站过渡。 事实上，控制躯干（“核心”）的位置和运动的能力对于在重量转移期间协调四肢的运动是必不可少的。 该项目的目标是将联合收割机的真实世界的肢体-核心动力学与加速度计通过商品手表和对侧臀部的手机收集的数据相结合，以提高对硬和软福尔斯的检测。 通过使用最先进的机器学习算法实时学习肢体-核心动态，将为每个用户创建个性化的跌倒风险分析和检测模型。 我们还将评估使用这项技术的老年患者的看法和偏好，并评估他们对持续收集数据和共享健康数据以改善健康的态度。 软件系统、真实世界的步态和重量转移运动以及相关的加速度计数据将免费提供给任何对机器学习研究健康状况以及跌倒风险和分析感兴趣的机构、研究人员或研究生。该项目将培训研究生和本科生的技术技能（机器学习，可穿戴技术和数据分析技能）以及与居住在长期护理设施中的老年人合作的人员技能。 虽然已经开发了许多结合人工智能（AI）和机器学习算法的跌倒检测设备，但该项目的重点是个性化跌倒风险检测。 该项目将探索使用与体重转移相关的老年人运动的运动学测量，以实现多任务和多模式机器学习算法来个性化跌倒风险检测。 开发了一种基于小样本的深度学习算法，该算法使用多任务和多模式学习框架优化以结合个体运动学特征。 其次，该团队将分析Azure Kinect系统捕获的运动转换，以确定加速度计数据与完整骨骼框架之间的关系，重点是肢体-核心动力学。 具体来说，我们的目标是确定生成对抗网络（GAN）是否可以用于从少量身体运动数据，智能手表和手机加速度数据中增加缺失的模态，这些数据直接从最有可能摔倒的老年参与者（即生活在辅助生活中心的人）那里收集。最后，我们将评估老年参与者对持续使用可穿戴设备进行跌倒风险分析和检测的看法和态度。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Progressive Cross-modal Knowledge Distillation for Human Action Recognition

• DOI: 10.1145/3503161.3548238
• 发表时间: 2022-08
• 期刊: Proceedings of the 30th ACM International Conference on Multimedia
• 作者: Jianyuan Ni;A. Ngu;Yan Yan-Yan

Personalized Watch-Based Fall Detection Using a Collaborative Edge-Cloud Framework

使用协作边缘云框架进行基于手表的个性化跌倒检测

• DOI: 10.1142/s0129065722500484
• 发表时间: 2022
• 期刊: International Journal of Neural Systems
• 影响因子: 8
• 作者: Ngu, Anne Hee;Metsis, Vangelis;Coyne, Shuan;Srinivas, Priyanka;Salad, Tarek;Mahmud, Uddin;Chee, Kyong Hee
• 通讯作者: Chee, Kyong Hee