Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
复制标题
DOI：
10.5555/1756006.1953039
复制
发表时间：
2010-03
期刊：
影响因子：
--
通讯作者：
Pascal Vincent;H. Larochelle;Isabelle Lajoie;Yoshua Bengio;Pierre-Antoine Manzagol
中科院分区：
文献类型：
--
作者：
Pascal Vincent;H. Larochelle;Isabelle Lajoie;Yoshua Bengio;Pierre-Antoine Manzagol
我们探索了一种构建深度网络的原始策略，该策略基于堆叠去噪自动编码器层，这些编码器在本地进行训练，以消除其输入的损坏版本。由此产生的算法是普通自动编码器堆叠的直接变化。然而，在分类问题的基准测试中，它显示出显著较低的分类错误，从而弥合了与深度信念网络（DBN）的性能差距，并在某些情况下超越了它。定性实验表明，与普通自动编码器相反，去噪自动编码器能够从自然图像块中学习类似Gabor的边缘检测器，并从数字图像中学习更大的笔画检测器。这项工作清楚地确立了使用去噪标准作为易于处理的无监督目标来指导有用的更高级别表示的学习的价值。
We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.