RI: Medium: From Vision to Dynamics

RI：媒介：从愿景到动态

• 批准号: 2312967
• 负责人: Robert Collins
• 金额: $ 120万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-15 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2312967&HistoricalAwards=false
• 关键词: RI; Medium; Vision; Dynamics

In the realms of health and sports, detailed analysis of human movement provides guidance for individualized performance assessment, training, and health evaluation. Data collection for biomechanical analysis of human motion is typically performed using expensive, specialized hardware that requires experiments to be carried out in a constrained lab setting. Computer vision algorithms that use video cameras for data collection enable observation in more natural environments with minimal cost, but current vision-based estimates of body pose and motion are not biomechanically accurate and lack a quantitative model of the physical forces underlying human movement. This research effort will train a modern machine learning system to perform imagery-based estimation of biomechanically accurate body pose, motion and force, turning informal computer vision into a scientific measurement tool for human movement and stability research in unconstrained, everyday settings using readily available low-cost video cameras. The results will enable novel applications in personalized training and preventative medicine, and lead to strategies for improving flexibility and stability among the elderly, reducing fall-induced mortality and improving quality of life, serving NSF's mission to promote progress of science to advance the national health. An outreach program covering topics in human and robot stability will inspire K-12 students with demonstrations that combine exercise, augmented reality, and humanoid robots.The project team will collect a unique, multi-modal human motion dataset and use it to train a novel deep learning system that combines multi-task regression with a biomechanical human motion model to estimate, from visual data alone, accurate pose and motion as well as dynamics such as muscle torques and ground reaction forces. The results will fill an existing gap between visual observation of kinematics and the inference of body dynamics, greatly improving the ability of computer vision to make inferences about balance and stability that require dynamics information. Data-driven discovery of discriminative features that are causal or correlated to stability will lead to identifying the most stability-enhancing poses/moves for a movement sequence and new measures for estimating stability of a person in motion. Guided by the scientific research, four applications will be explored: 1) Developing a vision-based stability surveillance system that can quantify and monitor changes in stability; 2) Developing a vision-based personal trainer for teaching stability-enhancing exercises; 3) Analyzing historical sports videos to reveal changes in performance styles and dynamics; and 4) Studying methods for retargeting human movements to a humanoid robot while maintaining stability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在健康和体育领域，对人类运动的详细分析为个性化绩效评估，培训和健康评估提供了指导。用于人类运动的生物力学分析的数据收集通常是使用昂贵的专业硬件进行的，该硬件需要在约束的实验室设置中进行实验。使用摄像机进行数据收集的计算机视觉算法可以在更为自然的环境中观察到最低的成本，但是当前的基于视觉的身体姿势和运动估计值不是生物力学上的，并且缺乏人类运动基本力量的物理力量的定量模型。这项研究工作将训练现代的机器学习系统，以对生物力学准确的身体姿势，运动和力量进行基于图像的估计，将非正式的计算机视觉转化为一种科学的测量工具，用于在未受约束的日常设置中使用易于使用的低成本摄像机进行人类运动和稳定性研究。 该结果将在个性化培训和预防医学中提供新的应用，并导致提高老年人的灵活性和稳定性的策略，从而降低了跌倒的死亡率和改善生活质量，从而服务NSF的使命，促进了促进科学进步以促进国民健康的发展。 涵盖人类和机器人稳定性主题的宣传计划将激发K-12学生的演示，将锻炼，增强现实和人形机器人结合起来。项目团队将收集一个独特的，多模式的人类运动数据集，并使用它来训练它的新型深度学习系统，以将多重任务回归与象征性的动作模型相结合，并构成效率和动作模型，以及效率的动作，以及效果，以及效果，以及效果，以及在效率上，以及效率的动作，以及效果，以及效率的效果，以及效率的效果，以及效率的效果，以及效果，以及效率的动作，以及效果，以及效率的动作，以及效率的效果。力量。结果将填补运动学视觉观察与身体动力学的推断之间的现有差距，从而大大提高了计算机视觉对需要动态信息的平衡和稳定性的推断的能力。数据驱动的因果关系或与稳定性相关的判别特征的发现将导致识别运动序列的最稳定性增强的姿势/移动，以及估计运动中人的稳定性的新措施。在科学研究的指导下，将探讨四个应用：1）开发一个基于视觉的稳定性监视系统，该系统可以量化和监视稳定性的变化； 2）开发一个基于远见的私人教练，用于教学稳定增强练习； 3）分析历史体育视频以揭示性能风格和动态的变化； 4）研究将人类运动重新定位到人形机器人的同时保持稳定性的方法。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准通过评估来支持的。