CAREER: Musculoskeletal Modeling with Wearable Sensors and Smartphone Cameras

这个学院早期职业发展(职业)奖的重点是使康复研究和治疗的步态分析民主化。流动性是人类健康的一个标志，然而，在近三分之一的美国人中，流动性限制继续降低个人独立性和整体生活质量。仅肌肉骨骼疾病一项就让美国经济损失了国内生产总值的5%。传统上，机动性研究是在步态实验室进行的，需要昂贵的设备、训练有素的人员和耗时的数据处理管道，这限制了研究的规模、设置和监测时间。计算机视觉和可穿戴传感的最新进展可能会将智能手机和可穿戴传感器的运动跟踪带到实验室、诊所和患者家中，成本微乎其微，使研究对象从数十个增加到数千个。然而，目前从视频和可穿戴设备进行运动跟踪的方法仍然不够准确，不能用于研究和临床翻译。其中一个挑战是，它们主要是由数据驱动的，依赖于有限的训练数据，这些数据不包括行动不便的患者。该项目将把人工智能(AI)和基于物理的建模的互补优势融合到一种新的运动跟踪范例中，该范例具有广泛的可访问性、动态健壮性和跨人类人口统计和能力的公平准确性。这个项目的技术目标是(1)创建和评估一个新的计算框架，用于使用可穿戴传感器和智能手机视频进行肌肉骨骼建模，以及(2)使用该框架回答有关循环加载对软骨健康的作用的基本问题。这项工作将从根本上改变生物力学工程师和康复专家用来回答有关行动限制的科学问题的数据量和类型，从而增强他们的能力。这些技术贡献将把多体动力学与最先进的机器学习模型相结合，使可穿戴传感器和智能手机摄像头能够进行准确的运动跟踪。此外，来自医学图像的特定于对象的解剖将被合并到这些建模框架中，以实现对内部生物力学的估计。这些科学贡献将通过产生以前无法获得的关于特定学科的关节机械如何在自然环境中调节软骨对循环负荷的反应的知识来推动矫形外科康复领域的发展，这将是设计用于预防骨关节炎的个性化康复技术的关键。该项目还将利用人工智能的科普吸引力和实用价值来激励、培训和留住下一代多元化的康复工程师。活动包括(1)与Facebook Reality Labs合作开展外联活动，让匹兹堡地区的K-12学生接触未来运动捕捉设施(Inspire)；(2)教育软件，协同生成注释数据集，以改进计算机视觉算法，同时向高中生传授值得信赖的人工智能(Train)；以及(3)促进高等教育公平和包容性的人工智能工具包(RETAIN)。该奖项反映了NSF的法定使命，通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) award focuses on democratizing gait analysis for rehabilitation research and therapy. Mobility is a hallmark of human health, yet mobility limitations continue to reduce personal independence and overall quality of life in nearly a third of Americans. Musculoskeletal conditions alone cost the United States’ economy 5% of its overall gross domestic product. Traditionally, mobility has been studied in gait laboratories, with expensive equipment, trained personnel, and time-consuming data processing pipelines, limiting studies in size, setting, and monitoring time. Recent advances in computer vision and wearable sensing could bring motion tracking with smartphones and wearable sensors to laboratories, clinics, and patient homes at negligible costs, growing studies from tens to thousands of subjects. Current approaches for motion tracking from videos and wearables, however, remain insufficiently accurate for research and clinical translation. One of the challenges is that they are primarily data-driven, relying on limited training data that do not include patients with mobility limitations. This project will merge the complementary strengths of artificial intelligence (AI) and physics-based modeling into a new motion-tracking paradigm that is widely accessible, dynamically robust, and equitably accurate across human demographics and abilities. The technical objectives of this project are to (1) create and evaluate a new computational framework for musculoskeletal modeling with wearable sensors and smartphone videos, and (2) use this framework to answer fundamental questions about the role of cyclic loading on cartilage health. This work will empower biomechanical engineers and rehabilitation specialists by fundamentally changing the amount and type of data they use to answer scientific questions around mobility limitations. The technical contributions will integrate multibody dynamics with state-of-the-art machine learning models to enable accurate motion tracking from wearable sensors and smartphone cameras. Additionally, subject-specific anatomy from medical images will be incorporated into these modeling frameworks to enable the estimation of internal biomechanics. The scientific contributions will advance the field of orthopaedic rehabilitation by generating previously unavailable knowledge on how subject-specific joint mechanics modulate cartilage response to cyclic loading in natural environments, which will be critical in the design of personalized rehabilitation technologies for osteoarthritis prevention. The project will also leverage the pop-science appeal and practical utility of AI to inspire, train, and retain the next generation of multidimensionally diverse rehabilitation engineers. Activities include (1) collaborative outreach events with Facebook Reality Labs to expose Pittsburgh area K-12 students to futuristic motion capture facilities (Inspire), (2) educational software that synergistically generates annotated datasets to improve computer-vision algorithms, while teaching high-school students about trustworthy AI (Train), and (3) an AI toolkit that promotes equity and inclusion in higher education (Retain).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.