Contrastive Learning for Diverse Disentangled Foreground Generation
Contrastive Learning for Diverse Disentangled Foreground Generation
复制标题
DOI：
10.1007/978-3-031-19787-1_19
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Yuheng Li;Yijun Li;Jingwan Lu;Eli Shechtman;Yong Jae Lee;Krishna Kumar Singh
中科院分区：
文献类型：
--
作者：
Yuheng Li;Yijun Li;Jingwan Lu;Eli Shechtman;Yong Jae Lee;Krishna Kumar Singh
我们介绍了一种新的方法，明确控制各种因素的不同的前景生成。现有的基于图像修补的前景生成方法通常难以生成多样化的结果，并且很少允许用户明确地控制特定的变化因素（例如，改变面部身份或面部修复结果的表情）。我们利用具有潜在代码的对比学习来为相同的掩码输入生成不同的前景结果。具体来说，我们定义了两组潜在代码，其中一组控制预定义的因素（“已知”），另一组控制其余的因素（“未知”）。来自两个集合的采样的潜码联合地对卷积核进行双调制，以引导生成器合成不同的结果。实验表明，我们的方法优于国家的最先进的结果多样性和生成可控性。
We introduce a new method for diverse foreground generation with explicit control over various factors. Existing image inpainting based foreground generation methods often struggle to generate diverse results and rarely allow users to explicitly control specific factors of variation (e.g., varying the facial identity or expression for face inpainting results). We leverage contrastive learning with latent codes to generate diverse foreground results for the same masked input. Specifically, we define two sets of latent codes, where one controls a pre-defined factor (“known”), and the other controls the remaining factors (“unknown”). The sampled latent codes from the two sets jointly bi-modulate the convolution kernels to guide the generator to synthesize diverse results. Experiments demonstrate the superiority of our method over state-of-the-arts in result diversity and generation controllability.