A Geometric Understanding of Deep Learning
A Geometric Understanding of Deep Learning
复制标题
对深度学习的几何理解
DOI：
10.1016/j.eng.2019.09.010
复制
发表时间：
2020-03-01
期刊：
影响因子：
12.8
通讯作者：
Gu, Xianfeng
中科院分区：
文献类型：
--
作者：
Lei, Na;An, Dongsheng;Gu, Xianfeng
本文介绍了生成对抗网络（GANs）的最优运输（OT）观点。自然数据集具有内在的模式，可以概括为流形分布原理：一类数据的分布接近于低维流形。gan主要完成流形学习和概率分布变换两个任务。后者可以使用经典的OT方法进行。从OT的角度来看，生成器计算OT映射，鉴别器计算生成的数据分布与真实数据分布之间的Wasserstein距离；两者都可以简化为一个凸几何优化过程。此外，OT理论还发现了产生器和鉴别器之间内在的协作关系，而不是竞争关系，以及模式崩溃的根本原因。我们还提出了一种新的生成模型，该模型使用自编码器（AE）进行流形学习，使用OT映射进行概率分布转换。该AE-OT模型提高了理论的严谨性和透明度，提高了计算的稳定性和效率；特别是，它消除了模态坍缩。实验结果验证了我们的假设，并证明了我们提出的模型的优越性。(c) 2020作者。由爱思唯尔有限公司代中国工程院高等教育出版社有限公司出版。
This work introduces an optimal transportation (OT) view of generative adversarial networks (GANs). Natural datasets have intrinsic patterns, which can be summarized as the manifold distribution principle: the distribution of a class of data is close to a low-dimensional manifold. GANs mainly accomplish two tasks: manifold learning and probability distribution transformation. The latter can be carried out using the classical OT method. From the OT perspective, the generator computes the OT map, while the discriminator computes the Wasserstein distance between the generated data distribution and the real data distribution; both can be reduced to a convex geometric optimization process. Furthermore, OT theory discovers the intrinsic collaborative-instead of competitive-relation between the generator and the discriminator, and the fundamental reason for mode collapse. We also propose a novel generative model, which uses an autoencoder (AE) for manifold learning and OT map for probability distribution transformation. This AE-OT model improves the theoretical rigor and transparency, as well as the computational stability and efficiency; in particular, it eliminates the mode collapse. The experimental results validate our hypothesis, and demonstrate the advantages of our proposed model. (C) 2020 THE AUTHORS. Published by Elsevier LTD on behalf of Chinese Academy of Engineering and Higher Education Press Limited Company.