Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk
复制标题
DOI：
10.1109/tit.2019.2935447
复制
发表时间：
2017-05
影响因子：
2.5
通讯作者：
Paul Hand;V. Voroninski
中科院分区：
文献类型：
--
作者：
Paul Hand;V. Voroninski
我们通过经验风险最小化研究生成深度神经网络提供的强制先验的理论特性。我们特别考虑两个模型，一个模型的任务是在给定最后一层访问权的情况下反转生成神经网络，另一个模型的任务是仅在给定最后一层的压缩线性观测的情况下反转生成神经网络。我们确定，在这两种情况下，在适当的网络层大小和网络权重的随机性假设下，经验风险最小化给出的非凸目标函数不具有任何虚假驻点。也就是说，我们以高概率确定，在远离所需解的两个标量倍数附近的小邻域的任何点，都存在下降方向。因此，在这些邻域之外不存在局部极小值、鞍点或其他驻点。这些结果构成了建立这些非凸优化问题的有利全局几何的第一个理论保证，并且它们弥合了强制执行深层生成先验的经验成功与对非线性逆问题的严格理解之间的差距。
We examine the theoretical properties of enforcing priors provided by generative deep neural networks via empirical risk minimization. In particular we consider two models, one in which the task is to invert a generative neural network given access to its last layer and another in which the task is to invert a generative neural network given only compressive linear observations of its last layer. We establish that in both cases, in suitable regimes of network layer sizes and a randomness assumption on the network weights, that the non-convex objective function given by empirical risk minimization does not have any spurious stationary points. That is, we establish that with high probability, at any point away from small neighborhoods around two scalar multiples of the desired solution, there is a descent direction. Hence, there are no local minima, saddle points, or other stationary points outside these neighborhoods. These results constitute the first theoretical guarantees which establish the favorable global geometry of these non-convex optimization problems, and they bridge the gap between the empirical success of enforcing deep generative priors and a rigorous understanding of non-linear inverse problems.