CAREER: Musculoskeletal Modeling with Wearable Sensors and Smartphone Cameras

该学院早期职业发展（Career）奖侧重于使步态分析在康复研究和治疗中民主化。行动能力是人类健康的标志，但行动能力的限制继续降低近三分之一的美国人的个人独立性和整体生活质量。仅肌肉骨骼疾病一项就使美国经济损失了其国内生产总值的5%。传统上，在步态实验室中研究移动性，需要昂贵的设备、训练有素的人员和耗时的数据处理管道，限制了研究的规模、设置和监测时间。计算机视觉和可穿戴传感技术的最新进展，可以让智能手机和可穿戴传感器的运动跟踪以微不足道的成本进入实验室、诊所和病人家中，使研究对象从数万人增加到数千人。然而，目前来自视频和可穿戴设备的运动跟踪方法对于研究和临床转化来说仍然不够准确。其中一个挑战是，它们主要是数据驱动的，依赖于有限的训练数据，不包括行动不便的患者。该项目将人工智能（AI）和基于物理的建模的互补优势融合到一个新的运动跟踪范式中，该范式可广泛访问，动态健壮，并且在人口统计和能力方面相当准确。该项目的技术目标是(1)使用可穿戴传感器和智能手机视频创建和评估用于肌肉骨骼建模的新计算框架，以及(2)使用该框架回答有关循环载荷对软骨健康作用的基本问题。这项工作将从根本上改变生物力学工程师和康复专家在回答有关活动能力限制的科学问题时使用的数据的数量和类型，从而增强他们的能力。技术贡献将集成多体动力学与最先进的机器学习模型，以实现可穿戴传感器和智能手机摄像头的精确运动跟踪。此外，来自医学图像的特定主题解剖将被纳入这些建模框架，以实现内部生物力学的估计。科学贡献将通过产生以前无法获得的关于特定主题关节力学如何调节软骨对自然环境中循环载荷的反应的知识来推进骨科康复领域，这将是设计个性化骨关节炎预防康复技术的关键。该项目还将利用人工智能的流行科学吸引力和实际效用来激励、培训和留住下一代多维康复工程师。活动包括(1)与Facebook Reality Labs的合作外展活动，向匹兹堡地区的K-12学生展示未来的动作捕捉设施（Inspire）；(2)协同生成注释数据集以改进计算机视觉算法的教育软件，同时向高中生传授可信赖的人工智能（Train）；(3)促进高等教育公平和包容的人工智能工具包（Retain）。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) award focuses on democratizing gait analysis for rehabilitation research and therapy. Mobility is a hallmark of human health, yet mobility limitations continue to reduce personal independence and overall quality of life in nearly a third of Americans. Musculoskeletal conditions alone cost the United States’ economy 5% of its overall gross domestic product. Traditionally, mobility has been studied in gait laboratories, with expensive equipment, trained personnel, and time-consuming data processing pipelines, limiting studies in size, setting, and monitoring time. Recent advances in computer vision and wearable sensing could bring motion tracking with smartphones and wearable sensors to laboratories, clinics, and patient homes at negligible costs, growing studies from tens to thousands of subjects. Current approaches for motion tracking from videos and wearables, however, remain insufficiently accurate for research and clinical translation. One of the challenges is that they are primarily data-driven, relying on limited training data that do not include patients with mobility limitations. This project will merge the complementary strengths of artificial intelligence (AI) and physics-based modeling into a new motion-tracking paradigm that is widely accessible, dynamically robust, and equitably accurate across human demographics and abilities. The technical objectives of this project are to (1) create and evaluate a new computational framework for musculoskeletal modeling with wearable sensors and smartphone videos, and (2) use this framework to answer fundamental questions about the role of cyclic loading on cartilage health. This work will empower biomechanical engineers and rehabilitation specialists by fundamentally changing the amount and type of data they use to answer scientific questions around mobility limitations. The technical contributions will integrate multibody dynamics with state-of-the-art machine learning models to enable accurate motion tracking from wearable sensors and smartphone cameras. Additionally, subject-specific anatomy from medical images will be incorporated into these modeling frameworks to enable the estimation of internal biomechanics. The scientific contributions will advance the field of orthopaedic rehabilitation by generating previously unavailable knowledge on how subject-specific joint mechanics modulate cartilage response to cyclic loading in natural environments, which will be critical in the design of personalized rehabilitation technologies for osteoarthritis prevention. The project will also leverage the pop-science appeal and practical utility of AI to inspire, train, and retain the next generation of multidimensionally diverse rehabilitation engineers. Activities include (1) collaborative outreach events with Facebook Reality Labs to expose Pittsburgh area K-12 students to futuristic motion capture facilities (Inspire), (2) educational software that synergistically generates annotated datasets to improve computer-vision algorithms, while teaching high-school students about trustworthy AI (Train), and (3) an AI toolkit that promotes equity and inclusion in higher education (Retain).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.