CIF: Small: Fast Stagewise Learning of Sparse Hierarchical Data Representations

现实世界的数据越来越多地具有许多维度或特征，但不是平等地填充所有维度，而是分布在内部维度低得多的表面上或附近的数据。例子包括fMRI数据和自然图像和视频数据集。 这项研究将推动目前可能在分析这样的大型数据集的边界，通过将层次结构纳入所谓的“稀疏字典表示”的数据。 研究结果将包括对机器学习方法的基本知识贡献，以及有助于调查复杂真实的世界数据的计算进步。 学习复杂数据结构的一个基本问题是如何有效地提取一组反映数据底层结构的特征。在许多应用中，包括人脸识别和物体识别，稀疏字典表示已被证明是有效的。然而，由于解决大规模稀疏表示问题是非常昂贵的，该方法通常限于中等规模的问题。本研究将该方法重新表述为一个渐进的、多阶段的、层次化的词典学习过程。这种方法从数据中增量地提取信息，并使用这些信息以有组织的分层方式细化数据表示。这使得能够以计算高效的方式构建大规模词典。因此，该方法将稀疏字典表示方法的能力扩展到更广泛的真实的世界应用。它还具有将现有的最先进的稀疏编码算法作为基本求解器的灵活性，因此可以将现有稀疏编码算法的功能扩展到多阶段、分层字典学习。

Increasingly, real-world data has many dimensions or features but rather than filling out all dimensions equally, the data distributed on or near a surface of much lower intrinsic dimension. Examples include fMRI data and natural image and video data sets. This research will push the boundary of what is currently possible in the analysis of such large data sets by incorporating hierarchical structure into what is known as a "sparse dictionary representation" of the data. The results will include both basic intellectual contributions to machine learning methods and computational advancements that will aid the investigation of complex real world data. A fundamental problem in learning the structure of complex data is how to effectively extract a set of features that reflects the underlying structure of the data. In many applications, including face recognition and object recognition, sparse dictionary representations have proved effective for this purpose. However, since solving large-scale sparse representation problems is very expensive, the method is generally limited to problems of moderate scale. This research reformulates the method into an incremental, multi-stage, hierarchical dictionary learning process. This approach incrementally extracts information from the data and uses this to refine the data representation in an organized hierarchical fashion. This enables the building of large-scale dictionaries in a computationally efficient way. The method hence extends the power of sparse dictionary representation methods to a wider variety of real world applications. It also has the flexibility to incorporate an existing state-of-the-art sparse coding algorithm as the basic solver and hence can extend the functionality of existing sparse coding algorithms to multi-stage, hierarchical dictionary learning.