Collaborative Research: CIF: Small: Deep Sparse Models: Analysis and Algorithms

深度卷积神经网络是一类数学模型，它提供了各种机器学习工具，取得了令人印象深刻的成功，经常在不同领域获得最先进的结果。然而，他们的理论理解和这些算法背后的基本思想仍然难以捉摸。这些问题对于认识和描述它们的局限性，为它们的性能提供保证，甚至开发和设计改进的实用模型都是必不可少的。获得这种理解的一个有希望的方法是对部署这些模型的样本类别做出假设（例如，使这些模型“足够简单”），目的是提供关于它们的理论见解。进一步理解这种“多层卷积稀疏模型”是本项目寻求完成的，拓宽了对其相关优化和学习问题的理解，并揭示了深度学习方法。该项目旨在推进不同层数的广义稀疏模型的最新进展，重点关注推理和学习问题。基于近端梯度法和次梯度下降法的新结果，将推导出与多层稀疏模型相关的反问题的可证明和有效的优化方法。这一建议将进一步把追求的拟订扩展到其他情况，增加参数选择和异常值的稳定性和稳健性。在此基础上，提出并分析了相应的无监督学习问题的有效算法。样本复杂度和泛化界限的问题将在监督学习环境中进行研究。在整个项目中，所得到的算法将根据它们与特定卷积网络架构的关系进行研究。该项目汇集了信号处理、字典学习、机器学习以及大规模问题优化方法的设计、分析和实施方面的专业知识。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep convolutional neural networks are a class of mathematical models that provide a variety of machine learning tools with impressive success, often obtaining state-of-the-art results across different fields. Yet, their theoretical understanding and the fundamental ideas behind these algorithms have remained elusive. These questions are essential to recognize and characterize their limitations, to provide guarantees for their performance, and even to develop and engineer improved practical models. A promising approach to obtain this understanding is to make assumptions about the class of samples on which these models are deployed (e.g., so that these are "simple enough") with the intention of providing theoretical insights about them. Further understanding of this 'multi-layered convolutional sparse model' is what this project seeks accomplish, broadening the understanding of its related optimization and learning problems, and shedding light on deep learning methodologies.This project proposes to advance the state of the art in generalized sparse models of different numbers of layers, focusing on both inference and learning problems. Provable and efficient optimization methods will be derived for the inverse problems associated with multilayer sparse models by relying on new results in proximal gradient and subgradient descent methods. This proposal will further extend the formulation of the pursuit to other settings, increasing stability and robustness to the choice of parameters and to outliers. Furthermore, efficient algorithms for the corresponding unsupervised learning problem will be proposed and analyzed. Questions of sample complexity and generalization bounds will in turn be studied in supervised learning settings. Throughout this project, the resulting algorithms will be studied in terms of their relation to specific convolutional network architectures. The project brings together combined expertise in signal processing, dictionary learning, machine learning, and the design, analysis and implementation of optimization methods for large-scale problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.