Real-time dynamic scene modeling and rendering

点击翻译按钮获取中文摘要

Imagine that you are sitting in your living room watching a live NHL game. You have a giant screen and a surround sound system, which make you feel as if you are rink-side. In some aspects, it may be better than watching rink-side -- you have the freedom to pause the game when you go for a drink and resume when you return, which is possible because newer television systems can locally store the signals and play them when you want. However, don't you also want to have the freedom to control your viewpoint so that you can watch the game from your favorite spot, or even freeze the action when someone scores and replay the scene from different perspectives? The proposed research program aims at making this a reality for the next generation of television systems.The objective of this research is to develop efficient methods that can capture the appearances of dynamic real scenes and present them interactively in 3D to local or remote viewers. The problem of dynamic scene modeling and rendering will be investigated from both image analysis and image synthesis points of view. On the analysis side, we will study how to efficiently and accurately estimate 3D locations and motions of objects in the scene. On the synthesis side, we will explore an efficient way to synthesize physically correct novel views based on sparsely sampled videos and the 3D information estimated. To achieve real-time performance, both image analysis and image synthesis algorithms will be designed and optimized for parallel execution on programmable graphics hardware. Our preliminary research in this area has yielded very promising results.