Learning overcomplete representations
Learning overcomplete representations
复制标题
DOI：
10.1162/089976600300015826
复制
发表时间：
2000-02-01
影响因子：
2.9
通讯作者：
Sejnowski, TJ
中科院分区：
文献类型：
--
作者：
Lewicki, MS;Sejnowski, TJ
在过完备基中，基向量的数量大于输入的维数，并且输入的表示不是基向量的唯一组合。过完备表示一直被提倡，因为它们在存在噪声的情况下具有更大的鲁棒性，可以更稀疏，并且在匹配数据中的结构时具有更大的灵活性。过完备码也被提出作为初级视觉皮层神经元的一些响应特性的模型。以前的工作集中在使用固定的过完备基（或字典）找到信号的最佳表示。我们提出了一种算法，学习过完备的基础，将其视为概率模型的观测数据。我们表明，过完备的基础可以产生一个更好的近似的数据的基本统计分布，从而可以导致更高的编码效率。这可以被看作是独立分量分析技术的推广，并提供了一种方法，贝叶斯重建的信号中存在的噪声和盲源分离时，有更多的来源比混合物。
In an overcomplete basis, the number of basis vectors is greater than the dimensionality of the input, and the representation of an input is not a unique combination of basis vectors. Overcomplete representations have been advocated because they have greater robustness in the presence of noise, can be sparser, and can have greater flexibility in matching structure in the data. Overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex. Previous work has focused on finding the best representation of a signal using a fixed overcomplete basis (or dictionary). We present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data. We show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency. This can be viewed as a generalization of the technique of independent component analysis and provides a method for Bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures.