Collaborative Research: Algorithms for Learning Regularizations of Inverse Problems with High Data Heterogeneity

在当今的大数据时代，大量的数据正在以不同的获取方式和不同的格式从不同的来源收集。这种高度的异质性给大数据的分析、推理、计算等诸多方面带来了严峻的挑战。该项目旨在开发新颖的建模和计算方法，包括高度结构化的深度神经网络和新颖的训练算法，以有效地解决这一具有挑战性的问题。该项目的结果将为涉及大型异构数据集的广泛重要领域提供强大的计算工具，例如信号处理，医学成像，计算机视觉和生物信息学。本课题的研究包括三个主要组成部分：(1)开发可学习优化算法（LOAs），该算法可导出求解非凸非光滑逆问题的高效方案。这些LOAs有效地将残差学习架构集成到精确和不精确下降型算法中，不仅在实践中与最先进的方法相比具有突出的效率，而且在理论上也有严格的收敛保证；(2)基于双层优化的学习loa参数的新颖训练策略，可以探索异构数据集中各种任务之间的潜在共同特征以及任务特定特征；(3)利用综合计算和采样复杂度分析解决参数训练的双层优化问题的有效方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In today's era of big data, massive amount of data are being collected in various acquisition settings and different formats from diverse sources. Such high heterogeneity has posed serious challenges in many aspects of analysis, inference, and computation involving big data. This project aims at developing novel modeling and computational methods, including highly structured deep neural networks and novel training algorithms, to effectively address this challenging issue. Results of this project will provide powerful computational tools to a broad range of important fields involving large heterogenous data sets, such as signal processing, medical imaging, computer vision, and bioinformatics.The research in this project includes three major components: (1) Development of learnable optimization algorithms (LOAs) which induce highly efficient schemes for solving nonconvex and nonsmooth inverse problems. These LOAs effectively integrate residual learning architectures into exact and inexact descent-type algorithms, which not only have outstanding efficiency compared to the state-of-the-art methods in practice but are also supported by rigorous convergence guarantees in theory; (2) Novel training strategies based on bi-level optimization to learn the parameters of the LOAs, which can explore the underlying common features across a variety of tasks in heterogeneous data sets as well as the task-specific features; and (3) Efficient methods to solve the bi-level optimization problems of parameter training with comprehensive computation and sampling complexity analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.