How Well Generative Adversarial Networks Learn Distributions
How Well Generative Adversarial Networks Learn Distributions
复制标题
DOI：
10.2139/ssrn.3714011
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Tengyuan Liang
中科院分区：
文献类型：
--
作者：
Tengyuan Liang
本文利用对抗框架和生成对抗网络（GAN）研究了学习分布的隐式收敛速度，其中Wasserstein，Sobolev，MMD GAN和广义/模拟矩法（GMM/SMM）是特例。我们研究了广泛的参数和非参数的目标分布，根据主机的客观评价指标。我们研究如何通过正则化的透镜获得GAN的良好统计保证。在非参数端，我们得到了最优的极小极大率分布估计下的对抗框架。在参数端，我们建立了一个一般神经网络类（包括深度泄漏ReLU网络）的理论，该理论描述了生成器和递归对选择的相互作用。我们发现并隔离了一个新的正则化概念，称为生成器-对-对正则化，它揭示了GAN与经典的参数和非参数方法相比显式分布估计的优势。我们开发了新的oracle不等式作为分析GAN的主要技术工具，这是独立的兴趣。
This paper studies the rates of convergence for learning distributions implicitly with the adversarial framework and Generative Adversarial Networks (GAN), which subsume Wasserstein, Sobolev, MMD GAN, and Generalized/Simulated Method of Moments (GMM/SMM) as special cases. We study a wide range of parametric and nonparametric target distributions, under a host of objective evaluation metrics. We investigate how to obtain a good statistical guarantee for GANs through the lens of regularization. On the nonparametric end, we derive the optimal minimax rates for distribution estimation under the adversarial framework. On the parametric end, we establish a theory for general neural network classes (including deep leaky ReLU networks), that characterizes the interplay on the choice of generator and discriminator pair. We discover and isolate a new notion of regularization, called the generator-discriminator-pair regularization, that sheds light on the advantage of GANs compared to classical parametric and nonparametric approaches for explicit distribution estimation. We develop novel oracle inequalities as the main technical tools for analyzing GANs, which is of independent interest.