Surround 3D video content creation with moving cameras

点击翻译按钮获取中文摘要

The overall goal of the project is to develop image acquisition and processing technology that will enable relatively low-cost generation of 3D surround video content, integrating live action footage with computer graphic imagery. In particular we will investigate the utility of a novel imaging system that consists of a single high resolution stereo pair of cameras combined with a fixed ring of lower resolution cameras. The imaging system will be mobile and able to track objects of interest, thereby keeping the stereo pair fixated on aspects of the scene relevant to the cinematic content being generated. The tracking capability, in addition to permitting a single stereo camera pair to be used, also minimizes motion blur that would otherwise arise with non-tracking cameras as the camera system is translated. We will investigate approaches to scripting of the image acquisition (camera tracking motions) and graphics content generation that will allow the content producer to bias the viewer's attention towards scene features relevant to the content's narrative, while still providing a sense of surround immersion.The project will also address finding solutions to a number of technical problems in the generation of 3D surround content. The first is to minimize image stitching defects arising from the multiple perspectives of the cameras. These generate parallax, which is important to provide depth cues for the viewings, but problematic when stitching images to form panoramas. We propose to implement techniques that are half-occlusion aware (half-occlusions are places where one eye sees scene details that the other eye does not) to inform the stitching process. We will also investigate the coupled generation of the CGI panoramas and the live action (camera) panoramas to minimize artifacts created during the compositing step.