Towards Disentangling Latent Space for Unsupervised Semantic Face Editing
Towards Disentangling Latent Space for Unsupervised Semantic Face Editing
复制标题
DOI：
10.1109/tip.2022.3142527
复制
发表时间：
2022-01-01
影响因子：
10.6
通讯作者：
Qiu, Guoping
中科院分区：
文献类型：
--
作者：
Liu, Kanglin;Cao, Gaofeng;Qiu, Guoping
在StyleGAN生成的图像中，面部属性在潜在空间中纠缠，这使得在不影响其他属性的情况下独立控制特定属性变得非常困难。监督属性编辑需要带注释的训练数据，这很难获得，并且将可编辑的属性限制在带有标签的属性上。因此，在解纠缠的潜在空间中进行无监督属性编辑是实现简洁、通用的语义人脸编辑的关键。本文提出了一种基于权值分解和正交正则化的结构-纹理无关架构（STIA-WO）来解决无监督语义人脸编辑的潜在空间问题。通过将STIA-WO应用于GAN，我们开发了一个StyleGAN，称为STGAN-WO，它通过利用风格向量构建一个完全可控的权重矩阵来调节图像合成，并采用正交正则化来确保风格向量的每个条目只控制一个独立的特征矩阵。为了进一步解除面部属性的纠缠，STGAN-WO引入了一种结构-纹理无关的架构，该架构利用两个独立且同分布（i.i.d）的潜在向量以一种解纠缠的方式控制纹理和结构成分的合成。无监督语义编辑是通过将粗层中的潜码沿其正交方向移动来改变纹理相关属性或通过改变细层中的潜码来操纵结构相关属性来实现的。实验结果表明，与现有的属性编辑方法相比，我们的STGAN-WO算法可以实现更好的属性编辑。
Facial attributes in StyleGAN generated images are entangled in the latent space which makes it very difficult to independently control a specific attribute without affecting the others. Supervised attribute editing requires annotated training data which is difficult to obtain and limits the editable attributes to those with labels. Therefore, unsupervised attribute editing in an disentangled latent space is key to performing neat and versatile semantic face editing. In this paper, we present a new technique termed Structure-Texture Independent Architecture with Weight Decomposition and Orthogonal Regularization (STIA-WO) to disentangle the latent space for unsupervised semantic face editing. By applying STIA-WO to GAN, we have developed a StyleGAN termed STGAN-WO which performs weight decomposition through utilizing the style vector to construct a fully controllable weight matrix to regulate image synthesis, and employs orthogonal regularization to ensure each entry of the style vector only controls one independent feature matrix. To further disentangle the facial attributes, STGAN-WO introduces a structure-texture independent architecture which utilizes two independently and identically distributed (i.i.d.) latent vectors to control the synthesis of the texture and structure components in a disentangled way. Unsupervised semantic editing is achieved by moving the latent code in the coarse layers along its orthogonal directions to change texture related attributes or changing the latent code in the fine layers to manipulate structure related ones. We present experimental results which show that our new STGAN-WO can achieve better attribute editing than state of the art methods.