Online Learning for Matrix Factorization and Sparse Coding
Online Learning for Matrix Factorization and Sparse Coding
复制标题
DOI：
10.5555/1756006.1756008
复制
发表时间：
2009-08
期刊：
影响因子：
--
通讯作者：
J. Mairal;F. Bach;J. Ponce;G. Sapiro
中科院分区：
文献类型：
--
作者：
J. Mairal;F. Bach;J. Ponce;G. Sapiro
稀疏编码-即将数据向量建模为基本元素的稀疏线性组合-广泛用于机器学习，神经科学，信号处理和统计学。本文主要研究大规模矩阵分解问题，该问题包括学习基集以使其适应特定数据。这个问题的变体包括信号处理中的字典学习，非负矩阵分解和稀疏主成分分析。在本文中，我们提出了一种新的在线优化算法来解决这些任务，该算法基于随机近似，可以优雅地扩展到具有数百万训练样本的大型数据集，并自然扩展到各种矩阵分解公式，使其适用于广泛的学习问题。收敛的证明，沿着与自然图像和基因组数据的实验表明，它导致国家的最先进的性能方面的速度和优化的小型和大型数据集。
Sparse coding--that is, modelling data vectors as sparse linear combinations of basis elements--is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set in order to adapt it to specific data. Variations of this problem include dictionary learning in signal processing, non-negative matrix factorization and sparse principal component analysis. In this paper, we propose to address these tasks with a new online optimization algorithm, based on stochastic approximations, which scales up gracefully to large data sets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems. A proof of convergence is presented, along with experiments with natural images and genomic data demonstrating that it leads to state-of-the-art performance in terms of speed and optimization for both small and large data sets.