Learning to generate line drawings that convey geometry and semantics
Learning to generate line drawings that convey geometry and semantics
复制标题
DOI：
10.1109/cvpr52688.2022.00776
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Caroline Chan;F. Durand;Phillip Isola
中科院分区：
文献类型：
--
作者：
Caroline Chan;F. Durand;Phillip Isola
本文提出了一种由照片生成线条画的非配对方法。当前的方法通常依赖于高质量的成对数据集来生成线条图。然而，这些数据集通常由于属于特定领域的绘图主题或收集的数据量而具有限制。尽管最近在无监督图像到图像翻译方面的工作取得了很大进展，但最新的方法仍然难以生成令人信服的线条画。我们观察到线条画是场景信息的编码，并试图传达3D形状和语义。我们将这些观察结果构建成一组目标，并训练图像翻译将照片映射到线条画中。我们引入了一个几何损失，预测深度信息的图像特征的线条画，和语义损失相匹配的CLIP功能的线条画与其相应的照片。我们的方法优于国家的最先进的不成对的图像翻译和线条画生成方法创建线条画从任意的照片。
This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, these datasets often have limitations due to the subjects of the drawings belonging to a specific domain, or in the amount of data collected. Although recent work in unsupervised image-to-image translation has shown much progress, the latest methods still struggle to generate compelling line drawings. We observe that line drawings are encodings of scene information and seek to convey 3D shape and semantic meaning. We build these observations into a set of objectives and train an image translation to map photographs into line drawings. We introduce a geometry loss which predicts depth information from the image features of a line drawing, and a semantic loss which matches the CLIP features of a line drawing with its corresponding photograph. Our approach outperforms state-of-the-art un-paired image translation and line drawing generation methods on creating line drawings from arbitrary photographs.