Collaborative Research: SCH: Personalized Watch-based Fall Risk Analysis and Detection with Cross Modal Learning

合作研究：SCH：通过跨模态学习进行基于手表的个性化跌倒风险分析和检测

• 批准号: 2123521
• 负责人: Yan Yan
• 金额: $ 19.35万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2123521&HistoricalAwards=false
• 关键词: Collaborative; Research; SCH; Personalized; Watch

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long-term care facilities. While numerous fall detection devices incorporating artificial intelligence (AI) and machine learning algorithms have been developed, this project focuses on personalizing fall risk detection. This project will explore the use of kinematic measurements of an elderly individual’s movements associated with weight transfer to enable multi-task and multi-modal machine learning algorithms to personalize fall risk detection. A small-sample-based deep learning algorithm optimized to incorporate individual kinematic characteristics using multi-task and multi-modal learning frameworks is developed. Second, the team will analyze the movement transitions captured by the Azure Kinect system in order to identify relationships between the accelerometer data and the complete skeletal frame with an emphasis on the limb-core dynamics. Specifically, our goal is to determine whether or not Generative Adversarial Networks (GANs) can be used to augment missing modality from a small amount of body motion data, smartwatch and phone acceleration data collected directly from elderly participants who are at most risk of falling, namely those living in an assisted living center. Finally, we will evaluate the perception and attitudes of the elderly participants towards the continuous use of wearable devices for fall risk analysis and detection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

跌倒是老年人发病和死亡的重要原因。一种用于估计跌倒风险和检测跌倒的强大且低成本的解决方案将使老年人能够独立生活，并降低因跌倒而产生的医疗成本。可穿戴设备已经被开发出来，用来检测“硬摔伤”，即导致受伤的摔伤。然而，很多老年人跌倒并不会造成身体伤害(“软性跌倒”)。这与转身和坐立转换等重量转移活动有关。事实上，控制躯干(“核心”)的位置和运动的能力对于在重量转移过程中协调肢体的运动是必不可少的。该项目的目标是将真实世界中个体的肢体核心动力学与加速度计通过商品手表和对侧臀部的手机收集的数据相结合，以提高对硬摔和软摔的检测。通过使用最先进的机器学习算法实时学习肢体-核心动力学，将为每个用户创建个性化的跌倒风险分析和检测模型。我们还将评估使用这项技术的老年患者的看法和偏好，并评估他们对持续收集数据和共享健康数据以改善健康的态度。该软件系统、真实世界的步态和重量转移运动以及相关的加速度计数据将免费提供给任何对机器学习研究健康状况以及跌倒风险和分析感兴趣的机构、调查员或研究生。该项目将培训研究生和本科生的技术技能(机器学习、可穿戴技术和数据分析技能)以及与长期护理设施中的老年人打交道的人际技能。虽然已经开发了许多结合了人工智能(AI)和机器学习算法的跌倒检测设备，但该项目的重点是个性化跌倒风险检测。该项目将探索使用与体重转移相关的老年人个人运动的运动学测量，以使多任务和多模式机器学习算法能够个性化摔倒风险检测。开发了一种基于小样本的深度学习算法，该算法利用多任务和多模式学习框架优化了个体的运动学特征。其次，该团队将分析Azure Kinect系统捕获的运动过渡，以确定加速度计数据和完整骨骼框架之间的关系，重点是肢体-核心动力学。具体地说，我们的目标是确定生成性对抗网络(GANS)是否可以用于从少量的身体运动数据、智能手表和手机加速数据中增强缺失的通道，这些数据直接从最有可能跌倒的老年参与者那里收集，即那些生活在辅助生活中心的人。最后，我们将评估老年参与者对持续使用可穿戴设备进行跌倒风险分析和检测的看法和态度。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Progressive Cross-modal Knowledge Distillation for Human Action Recognition

• DOI: 10.1145/3503161.3548238
• 发表时间: 2022-08
• 期刊: Proceedings of the 30th ACM International Conference on Multimedia
• 作者: Jianyuan Ni;A. Ngu;Yan Yan-Yan

Towards Saner Deep Image Registration

• DOI: 10.1109/iccv51070.2023.01145
• 发表时间: 2023-07
• 期刊: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Bin Duan;Ming Zhong;Yan Yan-Yan