CHS: Small: High Resolution Motion Capture

CHS：小：高分辨率运动捕捉

• 批准号: 2008564
• 负责人: Yin Yang
• 金额: $ 49.97万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008564&HistoricalAwards=false
• 关键词: CHS; Small; Resolution; Motion; Capture

This project studies a new type of full-body motion capture technology which has been enabled by recent advances in deep learning and high-resolution digital cameras. Unlike classical motion capture systems which rely on suits with small attached spheres that serve as markers, this project introduces a new type of suit using a special printed pattern instead of any attachments. This pattern will contain a new type of markers with two distinct advantages: (1) the ability to automatically detect which marker is which, and (2) a significantly more dense set of markers than previous systems. The proposed approach is fully passive and therefore easy to use, relying only (a) indoor lighting or natural daylight and (b) a suit made of elastic fabric with a special printed pattern, but without any wires or batteries. The only required electronics are standard digital cameras, ranging from just a few cameras to massive multi-camera systems. This flexibility will allow us to support applications on various scales, from individual researchers or makers to large institutions or production studios. New technology for full-body capture, featuring higher accuracy while being easy to use, has the potential to impact research and clinical studies of human motion, e.g., in orthopedics, sports medicine, rehabilitation, physical therapy and ergonomics. High-quality human motion data can also facilitate better virtual or augmented reality systems and applications. By combining computer science and human motion, motion capture systems enables unique educational and outreach opportunities involving activities popular among young people, such as sports and gymnastics. The idea of using a new type of motion capture suit with texture-based markers recognized using artificial neural networks has not been explored before and opens up many interesting research questions such as "Which types of markers and suit textures will lead to the best detection and labeling results, despite large elastic distortions induced by the motion of the skin?" The proposed computing methodology requires training and validation of neural networks and contributes to research on the following topics: (1) synthetic data generation, (2) automated data augmentation, (3) confidence calibration of neural networks, in particular teaching neural networks to quantify the risk of errors in their own predictions. To further improve robustness and accuracy, the probabilistic output of neural networks will be combined with priors, such as a 3D deformable shape model; this can be done in a principled way via Bayesian inference, which leads to research problems combining continuous and discrete optimization. Finally, the high-resolution data produced by the envisioned system motivates research on improving the anatomical realism of deformable shape models, in particular data-driven modeling of joint kinematics and muscle activations. The envisioned full-body capture system will be designed to be easy to replicate and deploy at various institutions, clinics or studios. This project will share research results through common open source codebase, facilitating collaboration and data sharing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目研究了一种新型的全身运动捕捉技术，该技术已通过深度学习和高分辨率数码相机的最新进展实现。与传统的动作捕捉系统不同，传统的动作捕捉系统依赖于带有小球体的套装作为标记，该项目引入了一种新型的套装，使用特殊的印刷图案而不是任何附件。该模式将包含具有两个明显优点的新型标记：（1）自动检测哪个标记是哪个标记的能力，以及（2）比先前系统显著更密集的标记集合。所提出的方法是完全被动的，因此易于使用，仅依赖于（a）室内照明或自然日光和（B）由具有特殊印刷图案的弹性织物制成的套装，但没有任何电线或电池。唯一需要的电子产品是标准的数码相机，从几个相机到大规模的多相机系统。这种灵活性将使我们能够支持各种规模的应用程序，从个人研究人员或制造商到大型机构或生产工作室。用于全身捕捉的新技术，具有更高的准确性，同时易于使用，有可能影响人体运动的研究和临床研究，例如，整形外科、运动医学、康复、物理治疗和人体工程学。高质量的人体运动数据还可以促进更好的虚拟或增强现实系统和应用。通过将计算机科学和人体运动相结合，运动捕捉系统能够提供独特的教育和推广机会，涉及年轻人中流行的活动，如体育和体操。使用新型动捕服的想法以前从未被探索过，该运动捕服具有使用人工神经网络识别的基于纹理的标记，并提出了许多有趣的研究问题，例如“哪些类型的标记和运动捕服纹理将导致最好的检测和标记结果，尽管皮肤运动引起了很大的弹性变形？“拟议的计算方法需要训练和验证神经网络，并有助于以下主题的研究：（1）合成数据生成，（2）自动数据增强，（3）神经网络的置信度校准，特别是教神经网络量化自己预测中的错误风险。为了进一步提高鲁棒性和准确性，神经网络的概率输出将与先验知识相结合，例如3D可变形形状模型;这可以通过贝叶斯推理以原则性的方式完成，从而导致研究问题结合连续和离散优化。最后，由所设想的系统产生的高分辨率数据激发了关于改进可变形形状模型的解剖真实性的研究，特别是关节运动学和肌肉激活的数据驱动建模。设想的全身捕捉系统将被设计为易于复制和部署在各种机构，诊所或工作室。该项目将通过共同的开源代码库共享研究成果，促进合作和数据共享。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。