SofGAN: A Portrait Image Generator with Dynamic Styling
SofGAN: A Portrait Image Generator with Dynamic Styling
复制标题
DOI：
10.1145/3470848
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Anpei Chen;Ruiyang Liu;Ling Xie;Zhang Chen;Hao Su;Jingyi Yu
中科院分区：
文献类型：
--
作者：
Anpei Chen;Ruiyang Liu;Ling Xie;Zhang Chen;Hao Su;Jingyi Yu
最近，生成对抗网络（GAN）已被广泛用于肖像图像生成。然而，在GAN学习的潜在空间中，不同的属性，如姿势，形状和纹理风格，通常是纠缠在一起的，这使得对特定属性的显式控制变得困难。为了解决这个问题，我们提出了一个SofGAN图像生成器，将肖像的潜在空间解耦为两个子空间：几何空间和纹理空间。从两个子空间采样的潜码分别馈送到两个网络分支，一个用于生成具有规范姿势的肖像的3D几何形状，另一个用于生成纹理。对齐的3D几何形状还带有语义部分分割，编码为语义占用字段（SOF）。SOF允许在任意视图上渲染一致的2D语义分割图，然后将其与生成的纹理图融合，并使用我们的语义实例模块将其风格化为肖像照片。通过大量的实验，我们表明，我们的系统可以生成高质量的人像图像与独立可控的几何和纹理属性。该方法在各种应用中也具有很好的推广性，例如外观一致的面部动画和动态造型。
Recently, Generative Adversarial Networks (GANs) have been widely used for portrait image generation. However, in the latent space learned by GANs, different attributes, such as pose, shape, and texture style, are generally entangled, making the explicit control of specific attributes difficult. To address this issue, we propose a SofGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space. The latent codes sampled from the two subspaces are fed to two network branches separately, one to generate the 3D geometry of portraits with canonical pose, and the other to generate textures. The aligned 3D geometries also come with semantic part segmentation, encoded as a semantic occupancy field (SOF). The SOF allows the rendering of consistent 2D semantic segmentation maps at arbitrary views, which are then fused with the generated texturemaps and stylized to a portrait photo using our semantic instance-wise module. Through extensive experiments, we show that our system can generate high-quality portrait images with independently controllable geometry and texture attributes. The method also generalizes well in various applications, such as appearance-consistent facial animation and dynamic styling.