SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting
SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting
复制标题
DOI：
10.1109/iccv48922.2021.01229
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
V. Jampani;Huiwen Chang;Kyle Sargent;Abhishek Kar;Richard Tucker;Michael Krainin;D. Kaeser;W. Freeman;D. Salesin;B. Curless;Ce Liu
中科院分区：
文献类型：
--
作者：
V. Jampani;Huiwen Chang;Kyle Sargent;Abhishek Kar;Richard Tucker;Michael Krainin;D. Kaeser;W. Freeman;D. Salesin;B. Curless;Ce Liu
单图像3D摄影使观众能够从新颖的视点观看静止图像。最近的方法将单目深度网络与图像网络相结合，以获得令人信服的结果。这些技术的一个缺点是使用硬深度分层，使他们无法模拟复杂的外观细节，如薄的头发状结构。我们提出SLIDE，一个模块化和统一的系统，用于单图像3D摄影，使用简单而有效的软分层策略，以更好地保留新视图中的外观细节。此外，我们提出了一种新颖的深度感知训练策略，为我们的inpainting模块，更适合3D摄影任务。由此产生的SLIDE方法是模块化的，可以使用其他组件，如分割和抠图来改进分层。同时，SLIDE采用高效的分层深度配方，只需要通过组件网络进行一次前向传递就可以生成高质量的3D照片。对三个视图合成数据集进行了广泛的实验分析，结合对野外图像集的用户研究，与现有的强基线相比，我们的技术具有优越的性能，同时在概念上简单得多。项目页面：https://varunjampani.github.io/slide
Single image 3D photography enables viewers to view a still image from novel viewpoints. Recent approaches combine monocular depth networks with inpainting networks to achieve compelling results. A drawback of these techniques is the use of hard depth layering, making them unable to model intricate appearance details such as thin hair-like structures. We present SLIDE, a modular and unified system for single image 3D photography that uses a simple yet effective soft layering strategy to better preserve appearance details in novel views. In addition, we propose a novel depth-aware training strategy for our inpainting module, better suited for the 3D photography task. The resulting SLIDE approach is modular, enabling the use of other components such as segmentation and matting for improved layering. At the same time, SLIDE uses an efficient layered depth formulation that only requires a single forward pass through the component networks to produce high quality 3D photos. Extensive experimental analysis on three view-synthesis datasets, in combination with user studies on in-the-wild image collections, demonstrate superior performance of our technique in comparison to existing strong baselines while being conceptually much simpler. Project page: https://varunjampani.github.io/slide