Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks
Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks
复制标题
DOI：
10.1145/2001269.2001295
复制
发表时间：
2011-10-01
影响因子：
22.7
通讯作者：
Ng, Andrew Y.
中科院分区：
文献类型：
--
作者：
Lee, Honglak;Grosse, Roger;Ng, Andrew Y.
人们对层次生成模型（如深度信念网络（dbn））的无监督学习非常感兴趣；然而，将这样的模型缩放到全尺寸的高维图像仍然是一个难题。为了解决这个问题，我们提出了卷积深度信念网络，这是一种分层生成模型，可以缩放到真实的图像尺寸。该模型具有平移不变性，支持高效的自底向上和自顶向下概率推理。我们方法的关键是概率最大池化，这是一种新颖的技术，可以以概率合理的方式缩小更高层的表示。我们的实验表明，该算法从未标记的物体图像和自然场景中学习有用的高级视觉特征，如物体部分。我们在几个视觉识别任务上展示了出色的性能，并表明我们的模型可以在全尺寸图像上执行分层（自下而上和自上而下）推理。
There has been much interest in unsupervised learning of hierarchical generative models such as deep belief networks (DBNs); however, scaling such models to full-sized, high-dimensional images remains a difficult problem. To address this problem, we present the convolutional deep belief network, a hierarchical generative model that scales to realistic image sizes. This model is translation-invariant and supports efficient bottom-up and top-down probabilistic inference. Key to our approach is probabilistic max-pooling, a novel technique that shrinks the representations of higher layers in a probabilistically sound way. Our experiments show that the algorithm learns useful high-level visual features, such as object parts, from unlabeled images of objects and natural scenes. We demonstrate excellent performance on several visual recognition tasks and show that our model can perform hierarchical (bottom-up and top-down) inference over full-sized images.