Full 3D Immersion: Capture and Visualization

点击翻译按钮获取中文摘要

This grant proposal is in the field of computer vision. Its primary focus is on developing new 3D immersion technologies, from the perspective of capturing reality accurately as well as visualization (large immersive projections, single virtual headset). The goals are simple. First, we intend to provide highly accurate, fast, and robust methods for capturing reality immersively (large field of view) and in 3D, without any constraint on the motion of camera or the scene observed. Second, we intend to provide highly efficient methods for large scale visualization of immersive 3D content (i.e. dome multi-projection, or other large surfaces), and for small scale visualization, such as virtual headsets (i.e. oculus rift). 3D Immersion is becoming more prevalent everyday, especially in the arts, entertainment, and video games. This is due mostly to recent advances in virtual headset technology such as the Oculus rift and to new interest toward 3D dome projection, where the screen is not only filling the observers fields of view, but also provides stereo images, so depth can be perceived accurately in all directions. Content creation for 3D immersion is mostly performed with 3D computer graphics, where it is easy to make stereo-immersive rendering. However, the accurate depiction of reality requires real world filming and has remained a challenge. Traditional "small field of view" stereo capture can not be used directly for immersion because it provides a single point of view while immersive capture requires capturing from all possible point of views simultaneously. The single person immersive 3D experience is also evolving rapidly in the form of new high performance virtual reality headsets. Software for viewing content has matured, especially with regards to 3D real-time graphics and video games. However, for immersive video, the large resolutions involved and geometry corrections (including parallax) still constitute a challenge. Our research program intends to explore 3D immersion, from the perspective of content creation and content presentation. We will develop new cameras for 3D spherical capture of reality, so it can be observed somewhere else with accurate depth perception in any direction, and in real-time. We will also develop better multi-projection, to simplify and automate its deployment while making it more perceptually accurate immersion with depth. Our long term goal is the "ultimate immersive 3D" experience, where the observer not only controls his gaze, but also his location. This new freedom for the observer will have a profound impact on the creation of 3D immersive content and make it much more challenging, thereby opening new avenues of research. Overall, if this research program is successful, we will have contributed to achieving the ultimate goal of 3D immersion: giving full freedom of movement to the observer, giving him a true felling of omnipresence.