Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
复制标题
DOI：
--
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Reinhard Heckel;Paul Hand
中科院分区：
文献类型：
--
作者：
Reinhard Heckel;Paul Hand
深度神经网络，特别是卷积神经网络，已经成为压缩图像和解决逆问题的高效工具，包括去噪，修复和从少量和嘈杂的测量中重建。这一成功可以部分归因于它们能够很好地表示和生成自然图像。与小波等经典工具相反，生成图像的深度神经网络具有大量参数-通常是其输出维度的倍数-并且需要在大型数据集上进行训练。在本文中，我们提出了一种未经训练的简单图像模型，称为深度解码器，它是一种深度神经网络，可以从很少的权重参数生成自然图像。深度解码器具有简单的架构，没有卷积，并且权重参数比输出维度少。这种欠参数化使深度解码器能够将图像压缩成一组简洁的网络权重，我们表明这与基于小波的阈值处理相当。此外，欠参数化为过拟合提供了障碍，允许深度解码器具有最先进的去噪性能。深度解码器很简单，因为每一层都有一个相同的结构，只包含一个上采样单元、通道的像素线性组合、ReLU激活和通道归一化。这种简单性使网络易于理论分析，并揭示了神经网络的各个方面，使它们能够形成有效的信号表示。
Deep neural networks, in particular convolutional neural networks, have become highly effective tools for compressing images and solving inverse problems including denoising, inpainting, and reconstruction from few and noisy measurements. This success can be attributed in part to their ability to represent and generate natural images well. Contrary to classical tools such as wavelets, image-generating deep neural networks have a large number of parameters---typically a multiple of their output dimension---and need to be trained on large datasets. In this paper, we propose an untrained simple image model, called the deep decoder, which is a deep neural network that can generate natural images from very few weight parameters. The deep decoder has a simple architecture with no convolutions and fewer weight parameters than the output dimensionality. This underparameterization enables the deep decoder to compress images into a concise set of network weights, which we show is on par with wavelet-based thresholding. Further, underparameterization provides a barrier to overfitting, allowing the deep decoder to have state-of-the-art performance for denoising. The deep decoder is simple in the sense that each layer has an identical structure that consists of only one upsampling unit, pixel-wise linear combination of channels, ReLU activation, and channelwise normalization. This simplicity makes the network amenable to theoretical analysis, and it sheds light on the aspects of neural networks that enable them to form effective signal representations.