Neural Rerendering in the Wild
Neural Rerendering in the Wild
复制标题
DOI：
10.1109/cvpr.2019.00704
复制
发表时间：
2019-04
期刊：
影响因子：
--
通讯作者：
Moustafa Meshry;Dan B. Goldman;S. Khamis;Hugues Hoppe;Rohit Pandey;Noah Snavely;Ricardo Martin-Brualla
中科院分区：
文献类型：
--
作者：
Moustafa Meshry;Dan B. Goldman;S. Khamis;Hugues Hoppe;Rohit Pandey;Noah Snavely;Ricardo Martin-Brualla
我们探讨了整个场景捕捉-记录，建模，并在不同的外观下重新渲染一个场景，如季节和一天中的时间。从旅游地标的互联网照片开始，我们应用传统的三维重建来注册照片和近似场景作为一个点云。对于每张照片，我们将场景点渲染到深度帧缓冲区中，并训练深度神经网络来学习这些初始渲染到实际照片的映射。该重渲染网络还将潜在外观向量和指示行人等瞬态对象的位置的语义掩码作为输入。该模型进行评估的几个数据集的公开可用的图像跨越广泛的照明条件。我们创建短视频，演示图像视点，外观和语义标签的逼真操作。我们还比较结果，从互联网上的照片现场重建先前的工作。
We explore total scene capture --- recording, modeling, and rerendering a scene under varying appearance such as season and time of day. Starting from Internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud. For each photo, we render the scene points into a deep framebuffer, and train a deep neural network to learn the mapping of these initial renderings to the actual photos. This rerendering network also takes as input a latent appearance vector and a semantic mask indicating the location of transient objects like pedestrians. The model is evaluated on several datasets of publicly available images spanning a broad range of illumination conditions. We create short videos that demonstrate realistic manipulation of the image viewpoint, appearance, and semantic labels. We also compare results to prior work on scene reconstruction from Internet photos.