Collaborative Research: Algorithms for Learning Regularizations of Inverse Problems with High Data Heterogeneity

在当今的大数据时代，大量的数据正在以各种采集设置和来自不同来源的不同格式收集。如此高的异质性在涉及大数据的分析、推理和计算的许多方面都提出了严峻的挑战。该项目旨在开发新的建模和计算方法，包括高度结构化的深度神经网络和新的训练算法，以有效地解决这一具有挑战性的问题。本项目的研究成果将为信号处理、医学成像、计算机视觉和生物信息学等涉及大量异质数据集的重要领域提供强有力的计算工具。本项目的研究包括三个主要部分：（1）开发可学习的优化算法（LOAs），它可以诱导高效的非凸和非光滑反问题求解方案。这些LOA有效地将残差学习结构集成到精确和不精确下降型算法中，不仅在实践中具有比最先进的方法更高的效率，而且在理论上也得到了严格的收敛保证;（2）基于双层优化学习LOA参数的新型训练策略，它可以探索跨异构数据集中的各种任务的潜在共同特征以及特定于任务的特征;（3）求解双-该奖项反映了NSF的法定使命，并通过使用基金会的学术价值和更广泛的影响评审标准。

In today's era of big data, massive amount of data are being collected in various acquisition settings and different formats from diverse sources. Such high heterogeneity has posed serious challenges in many aspects of analysis, inference, and computation involving big data. This project aims at developing novel modeling and computational methods, including highly structured deep neural networks and novel training algorithms, to effectively address this challenging issue. Results of this project will provide powerful computational tools to a broad range of important fields involving large heterogenous data sets, such as signal processing, medical imaging, computer vision, and bioinformatics.The research in this project includes three major components: (1) Development of learnable optimization algorithms (LOAs) which induce highly efficient schemes for solving nonconvex and nonsmooth inverse problems. These LOAs effectively integrate residual learning architectures into exact and inexact descent-type algorithms, which not only have outstanding efficiency compared to the state-of-the-art methods in practice but are also supported by rigorous convergence guarantees in theory; (2) Novel training strategies based on bi-level optimization to learn the parameters of the LOAs, which can explore the underlying common features across a variety of tasks in heterogeneous data sets as well as the task-specific features; and (3) Efficient methods to solve the bi-level optimization problems of parameter training with comprehensive computation and sampling complexity analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.