RI: Small: Uncertainty-driven Dynamic 3D Reconstruction

RI：小：不确定性驱动的动态 3D 重建

• 批准号: 1217797
• 负责人: Philippos Mordohai
• 金额: $ 36.8万
• 依托单位: Stevens Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-08-01 至 2016-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1217797&HistoricalAwards=false
• 关键词: RI; Small; Uncertainty; driven; Dynamic

This project develops technologies of dynamic 3D reconstruction with applications to broader areas, such as free-viewpoint video, markerless motion capture, special effects for 3D and conventional films, and augmented reality. While image and video-based 3D reconstruction of static scenes is well-understood and is among the most active research areas in computer vision, the current 3D reconstruction methods are not be able to reconstruct dynamic scenes containing non-rigidly moving people, animals or objects well. Furthermore, these methods are unable to self-assess their output. This research effort casts dense multi-view 3D reconstruction as an estimation problem with explicit uncertainty modeling distinguishing between geometric and correspondence uncertainty which are due to very different causes. Other innovations include the combined use of viewpoint-based and world-based processing with explicit and implicit representations and uncertainty-driven regularization.The outcomes of this project improve 3D reconstruction quality and reduce cost for the above applications which have broader impact on different research areas. Ongoing outreach efforts focus on improving Science, Technology, Engineering and Mathematics (STEM) education in several ways: by teaching high school students during the summer, by mentoring them as interns and by training graduate students working with high school STEM teachers. The project also includes a plan of creating the first publicly available dataset of multiple-view dynamic scenes with ground truth depth.

该项目开发了动态3D重建技术，并将其应用于更广泛的领域，如自由视点视频、无标记运动捕捉、3D和传统电影的特殊效果以及增强现实。虽然基于图像和视频的静态场景三维重建是计算机视觉领域中最活跃的研究领域之一，但现有的三维重建方法不能很好地重建包含非刚性运动的人、动物或物体的动态场景。此外，这些方法无法自我评估其产出。这项研究工作将密集多视点三维重建作为一个估计问题，使用显式不确定性建模，区分几何不确定性和对应不确定性，这是由于非常不同的原因造成的。其他创新包括结合使用基于视点和基于世界的处理，显式和隐式表示以及不确定性驱动的正则化。该项目的结果提高了3D重建质量，降低了上述应用的成本，对不同的研究领域具有更广泛的影响。正在进行的外展工作侧重于通过几种方式改善科学、技术、工程和数学(STEM)教育：在暑期教授高中生，指导他们实习，以及培训与高中STEM教师合作的研究生。该项目还包括创建第一个公开可用的具有地面真实深度的多视角动态场景数据集的计划。

项目成果

Controlling a robotic stereo camera under image quantization noise

在图像量化噪声下控制机器人立体相机

• DOI: 10.1177/0278364917735163
• 发表时间: 2017
• 期刊: The International Journal of Robotics Research
• 作者: Freundlich, Charles;Zhang, Yan;Zhu, Alex Zihao;Mordohai, Philippos;Zavlanos, Michael M
• 通讯作者: Zavlanos, Michael M