RI: Small: Uncertainty-driven Dynamic 3D Reconstruction

点击翻译按钮获取中文摘要

This project develops technologies of dynamic 3D reconstruction with applications to broader areas, such as free-viewpoint video, markerless motion capture, special effects for 3D and conventional films, and augmented reality. While image and video-based 3D reconstruction of static scenes is well-understood and is among the most active research areas in computer vision, the current 3D reconstruction methods are not be able to reconstruct dynamic scenes containing non-rigidly moving people, animals or objects well. Furthermore, these methods are unable to self-assess their output. This research effort casts dense multi-view 3D reconstruction as an estimation problem with explicit uncertainty modeling distinguishing between geometric and correspondence uncertainty which are due to very different causes. Other innovations include the combined use of viewpoint-based and world-based processing with explicit and implicit representations and uncertainty-driven regularization.The outcomes of this project improve 3D reconstruction quality and reduce cost for the above applications which have broader impact on different research areas. Ongoing outreach efforts focus on improving Science, Technology, Engineering and Mathematics (STEM) education in several ways: by teaching high school students during the summer, by mentoring them as interns and by training graduate students working with high school STEM teachers. The project also includes a plan of creating the first publicly available dataset of multiple-view dynamic scenes with ground truth depth.