Training sparse natural image models with a fast Gibbs sampler of an extended state space
Training sparse natural image models with a fast Gibbs sampler of an extended state space
复制标题
使用扩展状态空间的快速吉布斯采样器训练稀疏自然图像模型
DOI：
--
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
M. Bethge
中科院分区：
文献类型：
--
作者：
Lucas Theis;Jascha Narain Sohl;M. Bethge
我们提出了一种基于高效阻塞吉布斯采样器的新学习策略，用于稀疏过完备线性模型。特别强调统计图像建模，其中超完备模型在发现稀疏表示方面发挥了重要作用。我们的吉布斯采样器比通用采样方案更快，同时也不需要调整，因为它没有参数。使用吉布斯采样器和期望最大化的持久变体，我们能够从数据中提取潜在源的高度稀疏分布。当应用于自然图像时，我们的算法学习类似于尖峰和平板分布的源分布。我们评估可能性并定量比较过完备线性模型与其完全对应模型以及专家模型的产物，这代表了完整线性模型的另一种过完备推广。与之前的说法相反，我们发现过完备的表示会带来显着的改进，但过完备的线性模型仍然表现不佳。
We present a new learning strategy based on an efficient blocked Gibbs sampler for sparse overcomplete linear models. Particular emphasis is placed on statistical image modeling, where overcomplete models have played an important role in discovering sparse representations. Our Gibbs sampler is faster than general purpose sampling schemes while also requiring no tuning as it is free of parameters. Using the Gibbs sampler and a persistent variant of expectation maximization, we are able to extract highly sparse distributions over latent sources from data. When applied to natural images, our algorithm learns source distributions which resemble spike-and-slab distributions. We evaluate the likelihood and quantitatively compare the performance of the overcomplete linear model to its complete counterpart as well as a product of experts model, which represents another overcomplete generalization of the complete linear model. In contrast to previous claims, we find that overcomplete representations lead to significant improvements, but that the overcomplete linear model still underperforms other models.