Collaborative Research: Sparse Optimization for Machine Learning and Image/Signal Processing

涉及信息技术、纳米技术、生物技术、民用基础设施、环境科学和其他重要领域的数据集往往非常大。由于数据量和相关模型尺寸的不断增长，对更有能力的数据处理模型的需求不断增加。应用程序中的数据集通常具有某些嵌入的稀疏性结构，也就是说，它们的本质特征可以用更少量的信息来表示。在这种观察的激励下，这个项目的目的是通过探索数据集中嵌入的稀疏结构来开发计算效率高的非光滑和非凸优化方法。研究人员预计该项目的成果将在许多应用领域得到应用。该项目的研究和相关教育内容预计将为本科生和研究生提供严格的培训，使他们具备应对大数据时代科学和技术挑战所需的技能。该项目将解决一系列应用（包括机器学习和稀疏图像/信号恢复）中数据稀疏建模导致的非平滑、非凸优化中的几个关键问题。对于学习和图像/信号恢复，将开发适当的稀疏正则化模型。将设计合适的稀疏性促进函数用于形成正则化项，并构造合适的基/变换，以便得到的正则化算法迫使其解是稀疏的。对于机器学习，将建立适当的再现核巴拿赫空间，以允许数据的复杂和丰富的几何和拓扑结构的表示，从而导致改进的学习结果。将在这些空间中建立所得到的学习方法的表示定理，以便即使假设空间是无限维的，也可以将学习解表示为有限数量的核会话与训练中使用的数据点的数量相等的组合。这些空间的几何特征将被用来诱导学习解的稀疏性。通过机器学习从数据中构建碱基或变换，有望改进图像/信号恢复方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data sets involved in information technology, nanotechnology, biotechnology, civil infrastructure, environmental science, and other important areas are often extremely large. Due to growing quantities of data and related model sizes, demands for more competent data processing models continue to increase. Data sets in applications often have certain embedded sparsity structures, in the sense that their essential intrinsic characteristics can be represented by smaller amounts of information. Motivated by this observation, the aim of this project is to develop computationally efficient methods for non-smooth and non-convex optimization by exploring sparsity structures embedded in data sets. The investigators anticipate that the outcomes of this project will be of use in many application areas. The research and associated educational components in this project are expected to provide undergraduate and graduate students rigorous training so that they will have the skill sets needed to face the scientific and technological challenges of the big data era. This project will address several critical issues in non-smooth, non-convex optimization that result from sparse modeling of data in a range of applications, including machine learning and sparse image/signal restoration. For both learning and image/signal restoration, proper sparse regularization models will be developed. Suitable sparsity promoting functions will be designed for use in forming regularization terms, and appropriate bases/transforms will be constructed so that the resulting regularization algorithms compel their solutions to be sparse. For machine learning, appropriate reproducing kernel Banach spaces will be built up to allow for the representation of complex and rich geometric and topological structures of data and hence lead to improved learning outcomes. Representer theorems of the resulting learning methods in these spaces will be established so that the learning solutions can be expressed as a combination of a finite number of kernel sessions with the number equal to that of the data points used in training even though the hypothesis space is of infinite dimension. Geometric features of these spaces will be employed to induce sparsity of the learning solutions. The construction of bases or transforms via machine learning from data is expected to lead to improved methods for image/signal restoration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.