Inverting Deep Generative models, One layer at a time
Inverting Deep Generative models, One layer at a time
复制标题
DOI：
--
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Qi Lei;A. Jalal;I. Dhillon;A. Dimakis
中科院分区：
文献类型：
--
作者：
Qi Lei;A. Jalal;I. Dhillon;A. Dimakis
我们研究了用ReLU激活来反演深度生成模型的问题。反演对应于找到尽可能多地解释观察到的测量的潜在码向量。在大多数以前的工作，这是通过尝试解决一个非凸优化问题，涉及发电机。在本文中，我们获得了一些新的理论结果的反演问题。我们表明，对于可实现的情况下，单层反演可以完全在多项式时间内，通过求解线性规划。此外，我们表明，对于多层，反演是NP-困难的，前图像集可以是非凸的。对于任意深度的生成模型，我们证明了如果层扩展并且随机选择权重，则精确恢复在多项式时间内以高概率是可能的。最近的工作分析了梯度下降反演的相同问题。他们的分析需要显着更高的扩展（对数的潜在维度），而我们提出的算法可以证明重建，即使是常数因子扩展。我们还提供了可证明的误差界重建噪声观测不同的规范。我们的经验验证表明，我们得到更好的重建时，潜在的维度是大的。
We study the problem of inverting a deep generative model with ReLU activations. Inversion corresponds to finding a latent code vector that explains observed measurements as much as possible. In most prior works this is performed by attempting to solve a non-convex optimization problem involving the generator. In this paper we obtain several novel theoretical results for the inversion problem. We show that for the realizable case, single layer inversion can be performed exactly in polynomial time, by solving a linear program. Further, we show that for multiple layers, inversion is NP-hard and the pre-image set can be non-convex. For generative models of arbitrary depth, we show that exact recovery is possible in polynomial time with high probability, if the layers are expanding and the weights are randomly selected. Very recent work analyzed the same problem for gradient descent inversion. Their analysis requires significantly higher expansion (logarithmic in the latent dimension) while our proposed algorithm can provably reconstruct even with constant factor expansion. We also provide provable error bounds for different norms for reconstructing noisy observations. Our empirical validation demonstrates that we obtain better reconstructions when the latent dimension is large.