Surround 3D video content creation with moving cameras

使用移动摄像机创建环绕 3D 视频内容

• 批准号: 501466-2016
• 负责人: Clark, James
• 金额: $ 2.33万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=647768
• 关键词: Surround; 3D; video; content; creation

The overall goal of the project is to develop image acquisition and processing technology that will enable relatively low-cost generation of 3D surround video content, integrating live action footage with computer graphic imagery. In particular we will investigate the utility of a novel imaging system that consists of a single high resolution stereo pair of cameras combined with a fixed ring of lower resolution cameras. The imaging system will be mobile and able to track objects of interest, thereby keeping the stereo pair fixated on aspects of the scene relevant to the cinematic content being generated. The tracking capability, in addition to permitting a single stereo camera pair to be used, also minimizes motion blur that would otherwise arise with non-tracking cameras as the camera system is translated. We will investigate approaches to scripting of the image acquisition (camera tracking motions) and graphics content generation that will allow the content producer to bias the viewer's attention towards scene features relevant to the content's narrative, while still providing a sense of surround immersion.****The project will also address finding solutions to a number of technical problems in the generation of 3D surround content. The first is to minimize image stitching defects arising from the multiple perspectives of the cameras. These generate parallax, which is important to provide depth cues for the viewings, but problematic when stitching images to form panoramas. We propose to implement techniques that are half-occlusion aware (half-occlusions are places where one eye sees scene details that the other eye does not) to inform the stitching process. We will also investigate the coupled generation of the CGI panoramas and the live action (camera) panoramas to minimize artifacts created during the compositing step.****

该项目的总体目标是开发图像采集和处理技术，以相对低成本生成 3D 环绕视频内容，将实景镜头与计算机图形图像集成。特别是，我们将研究一种新颖的成像系统的实用性，该系统由一对高分辨率立体相机与固定的低分辨率相机环组成。成像系统将是移动的并且能够跟踪感兴趣的对象，从而使立体对固定在与正在生成的电影内容相关的场景的各个方面。除了允许使用单个立体相机对之外，跟踪功能还可以最大限度地减少运动模糊，否则当相机系统平移时，非跟踪相机可能会出现运动模糊。我们将研究图像采集（摄像机跟踪运动）和图形内容生成的脚本方法，使内容制作者能够将观看者的注意力转移到与内容叙述相关的场景特征上，同时仍然提供环绕声沉浸感。****该项目还将解决 3D 环绕声内容生成中的许多技术问题的解决方案。首先是尽量减少由于摄像机的多个视角而产生的图像拼接缺陷。这些会产生视差，这对于为观看提供深度提示很重要，但在拼接图像形成全景图时会出现问题。我们建议实施半遮挡感知技术（半遮挡是一只眼睛看到场景细节而另一只眼睛看不到的地方）来通知拼接过程。我们还将研究 CGI 全景图和实景（摄像机）全景图的耦合生成，以最大程度地减少合成步骤中产生的伪影。****