Collaborative Research: Algorithms for Learning Regularizations of Inverse Problems with High Data Heterogeneity

在当今的大数据时代，海量数据正以各种获取环境和不同格式从不同来源收集。如此高的异构性给涉及大数据的分析、推理和计算的许多方面带来了严峻的挑战。该项目旨在开发新的建模和计算方法，包括高度结构化的深度神经网络和新的训练算法，以有效地解决这一具有挑战性的问题。该项目的研究成果将为信号处理、医学成像、计算机视觉和生物信息学等涉及大数据集的重要领域提供强大的计算工具。本项目的研究包括三个主要部分：(1)开发可学习优化算法(LOAS)，为解决非凸非光滑反问题提供高效的方案。这些LOAS有效地将残差学习结构集成到精确和不精确下降型算法中，不仅在实践上比最先进的方法具有卓越的效率，而且在理论上得到了严格的收敛保证：(2)基于双层优化的新颖训练策略来学习LOAS的参数，该训练策略可以探索各种不同数据集中各种任务的潜在共同特征以及任务特定特征；以及(3)通过全面的计算和采样复杂性分析来解决参数训练的双层优化问题的有效方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In today's era of big data, massive amount of data are being collected in various acquisition settings and different formats from diverse sources. Such high heterogeneity has posed serious challenges in many aspects of analysis, inference, and computation involving big data. This project aims at developing novel modeling and computational methods, including highly structured deep neural networks and novel training algorithms, to effectively address this challenging issue. Results of this project will provide powerful computational tools to a broad range of important fields involving large heterogenous data sets, such as signal processing, medical imaging, computer vision, and bioinformatics.The research in this project includes three major components: (1) Development of learnable optimization algorithms (LOAs) which induce highly efficient schemes for solving nonconvex and nonsmooth inverse problems. These LOAs effectively integrate residual learning architectures into exact and inexact descent-type algorithms, which not only have outstanding efficiency compared to the state-of-the-art methods in practice but are also supported by rigorous convergence guarantees in theory; (2) Novel training strategies based on bi-level optimization to learn the parameters of the LOAs, which can explore the underlying common features across a variety of tasks in heterogeneous data sets as well as the task-specific features; and (3) Efficient methods to solve the bi-level optimization problems of parameter training with comprehensive computation and sampling complexity analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.