RI: Medium: From Vision to Dynamics

在健康和运动领域，对人体运动的详细分析为个性化的表现评估、训练和健康评估提供了指导。用于人体运动的生物力学分析的数据收集通常使用昂贵的专用硬件来执行，这些硬件需要在受限的实验室环境中进行实验。使用摄像机进行数据收集的计算机视觉算法能够以最小的成本在更自然的环境中进行观察，但目前对身体姿势和运动的基于视觉的估计在生物力学上并不准确，并且缺乏人体运动背后的物理力的定量模型。这项研究工作将训练一个现代机器学习系统，对生物力学上准确的身体姿势、运动和力量进行基于图像的估计，将非正式的计算机视觉变成一种科学的测量工具，用于在不受约束的日常环境中使用现成的低成本摄像机进行人体运动和稳定性研究。 研究结果将使个性化训练和预防医学的新应用成为可能，并导致改善老年人灵活性和稳定性的策略，减少跌倒引起的死亡率和提高生活质量，为NSF的使命服务，以促进科学进步，促进国民健康。 一个涵盖人类和机器人稳定性主题的推广计划将通过结合联合收割机运动、增强现实和人形机器人的演示来激励K-12学生。项目团队将收集一个独特的多模态人体运动数据集，并使用它来训练一个新颖的深度学习系统，该系统将多任务回归与生物力学人体运动模型相结合，仅从视觉数据来估计，精确的姿势和运动以及肌肉扭矩和地面反作用力等动力学特性。这些结果将填补视觉观察运动学和推断身体动力学之间的现有空白，大大提高计算机视觉对需要动力学信息的平衡和稳定性进行推断的能力。与稳定性有因果关系或相关的区别性特征的数据驱动发现将导致识别运动序列的最稳定性增强姿势/移动以及用于估计运动中的人的稳定性的新措施。在科学研究的指导下，将探索四个应用：1）开发一个基于视觉的稳定性监测系统，可以量化和监测稳定性的变化; 2）开发一个基于视觉的私人教练，用于教授增强稳定性的练习; 3）分析历史体育视频，以揭示表现风格和动态的变化;以及4）研究在保持稳定性的同时将人类运动重定向到人形机器人的方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估而被认为值得支持。

In the realms of health and sports, detailed analysis of human movement provides guidance for individualized performance assessment, training, and health evaluation. Data collection for biomechanical analysis of human motion is typically performed using expensive, specialized hardware that requires experiments to be carried out in a constrained lab setting. Computer vision algorithms that use video cameras for data collection enable observation in more natural environments with minimal cost, but current vision-based estimates of body pose and motion are not biomechanically accurate and lack a quantitative model of the physical forces underlying human movement. This research effort will train a modern machine learning system to perform imagery-based estimation of biomechanically accurate body pose, motion and force, turning informal computer vision into a scientific measurement tool for human movement and stability research in unconstrained, everyday settings using readily available low-cost video cameras. The results will enable novel applications in personalized training and preventative medicine, and lead to strategies for improving flexibility and stability among the elderly, reducing fall-induced mortality and improving quality of life, serving NSF's mission to promote progress of science to advance the national health. An outreach program covering topics in human and robot stability will inspire K-12 students with demonstrations that combine exercise, augmented reality, and humanoid robots.The project team will collect a unique, multi-modal human motion dataset and use it to train a novel deep learning system that combines multi-task regression with a biomechanical human motion model to estimate, from visual data alone, accurate pose and motion as well as dynamics such as muscle torques and ground reaction forces. The results will fill an existing gap between visual observation of kinematics and the inference of body dynamics, greatly improving the ability of computer vision to make inferences about balance and stability that require dynamics information. Data-driven discovery of discriminative features that are causal or correlated to stability will lead to identifying the most stability-enhancing poses/moves for a movement sequence and new measures for estimating stability of a person in motion. Guided by the scientific research, four applications will be explored: 1) Developing a vision-based stability surveillance system that can quantify and monitor changes in stability; 2) Developing a vision-based personal trainer for teaching stability-enhancing exercises; 3) Analyzing historical sports videos to reveal changes in performance styles and dynamics; and 4) Studying methods for retargeting human movements to a humanoid robot while maintaining stability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.