Understanding Estimation and Generalization Error of Generative Adversarial Networks
Understanding Estimation and Generalization Error of Generative Adversarial Networks
复制标题
DOI：
10.1109/tit.2021.3053234
复制
发表时间：
2021-05
影响因子：
2.5
通讯作者：
Kaiyi Ji;Yi Zhou;Yingbin Liang
中科院分区：
文献类型：
--
作者：
Kaiyi Ji;Yi Zhou;Yingbin Liang
本文研究了生成对抗网络（GAN）训练的估计和泛化误差。在统计方面，我们为训练GAN的估计误差开发了一个上限和一个最小最大下限。上界结合了GAN的递归和生成器的作用，并在样本大小和ReLU激活下神经网络参数矩阵的范数方面匹配Minimax下界。在算法方面，我们为训练GAN的随机梯度方法（SGM）开发了一个泛化误差界。这样的界限证明了GAN在多次通过数据之后通过SGM训练的泛化能力，并反映了训练器和生成器之间的相互作用。我们的研究结果表明，生成器的训练需要更多的样本比训练的神经网络。这与经验观察是一致的，即训练的迭代器通常比生成器收敛得更快。实验结果验证了理论分析的正确性。
This article investigates the estimation and generalization errors of the generative adversarial network (GAN) training. On the statistical side, we develop an upper bound as well as a minimax lower bound on the estimation error for training GANs. The upper bound incorporates the roles of both the discriminator and the generator of GANs, and matches the minimax lower bound in terms of the sample size and the norm of the parameter matrices of neural networks under ReLU activation. On the algorithmic side, we develop a generalization error bound for the stochastic gradient method (SGM) in training GANs. Such a bound justifies the generalization ability of the GAN training via SGM after multiple passes over the data and reflects the interplay between the discriminator and the generator. Our results imply that the training of the generator requires more samples than the training of the discriminator. This is consistent with the empirical observation that the training of the discriminator typically converges faster than that of the generator. The experiments validate our theoretical results.