Towards a Better Global Loss Landscape of GANs
Towards a Better Global Loss Landscape of GANs
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Ruoyu Sun;Tiantian Fang;A. Schwing
中科院分区：
文献类型：
--
作者：
Ruoyu Sun;Tiantian Fang;A. Schwing
对GAN培训的了解仍然非常有限。一个主要的挑战是它的非凸-非凹的最小-最大目标，这可能导致次优的局部最小值。在这项工作中，我们对GAN的经验损失进行了全球景观分析。我们证明了一类可分离的GAN，包括原来的JS-GAN，有指数的坏盆被视为模式崩溃。我们还研究了相对论性配对GAN（RpGAN）损失耦合生成的样本和真实的样本。我们证明了RpGAN没有坏盆。对合成数据的实验表明，预测的坏盆确实可以出现在训练中。我们还进行了实验来支持我们的理论，即RpGAN比可分离GAN具有更好的景观。例如，我们的经验表明，RpGAN在相对狭窄的神经网络中比可分离GAN表现得更好。代码可以在这个https URL上找到。
Understanding of GAN training is still very limited. One major challenge is its non-convex-non-concave min-max objective, which may lead to sub-optimal local minima. In this work, we perform a global landscape analysis of the empirical loss of GANs. We prove that a class of separable-GAN, including the original JS-GAN, has exponentially many bad basins which are perceived as mode-collapse. We also study the relativistic pairing GAN (RpGAN) loss which couples the generated samples and the true samples. We prove that RpGAN has no bad basins. Experiments on synthetic data show that the predicted bad basin can indeed appear in training. We also perform experiments to support our theory that RpGAN has a better landscape than separable-GAN. For instance, we empirically show that RpGAN performs better than separable-GAN with relatively narrow neural nets. The code is available at this https URL.