Sparsity Regularization for Inverse Problems -- Theory, Algorithm and Application

许多科学和工程学科的一项基本任务是探索我们周围的世界，通常是以推断物理定律或从实验数据中确定其中的参数的形式。这就产生了各种各样的反问题，如从测量的响应中推断观察到的物理现象的原因或确定有关物体的介质性质。基于模型的反问题方法假设未知物理量与测量响应之间的已知关系，以线性或非线性方程的形式存在。逆问题在数值上具有挑战性，因为它们在数据噪声方面是不稳定的。最成功和最强大的技术之一是通过正则化的方式整合先验知识。本项目主要研究一种基于稀疏性约束的正则化技术。在稀疏性正则化中，人们寻找具有尽可能少的非零项（或小支持）的近似解。也就是说，我们的目的是通过简单地使用少量参数来解释物理现象。在这个项目中，我们研究了该方法的数学理论、计算技术和实际应用，特别是通过在寻求解上使用非凸惩罚。文献中提出了几种非凸处罚，特别是在统计学和机器学习中，我们将考虑，例如，流行的10处罚，桥处罚，平滑剪裁绝对偏差，最小最大凹度处罚和剪裁l1处罚。尽管它们很受欢迎，但它们在逆问题中的应用和研究仍然非常有限。提出的项目在反问题理论的框架内检查这些技术。具体而言，我们在项目期间重点关注以下目标：(a)开发一种有效求解正则化模型（具有非凸惩罚）的原始对偶活动集类型的计算方法，并严格建立算法的收敛性；(b)制定适用于正则化参数的选择规则，并分析“局部”最小化器的结构特性；(3)将非凸方法与自适应有限元法相结合，应用于断层成像。由于这些非凸模型有其起源和动机，因此要开发的计算技术，原始对偶活动集算法，广泛适用于许多其他领域，特别是机器学习和统计学。在数学上，稀疏正则化理论将为非光滑正则化的分析提供有价值的启示，非光滑正则化与成像和信号处理中的许多重要数学模型具有共同的结构。具体的正则化参数选择规则将使参数的自动选择具有严格的理论依据，提高目前基于繁琐试错的实践效率。断层成像的研究，即非凸惩罚和自适应算法的应用，将直接影响到医学成像领域。有一大批从事断层成像研究的研究人员将直接受益于这项研究，所获得的研究成果将不断地呈现给他们。更一般地说，它为开发微分方程非线性反问题的有效算法提供了指导。总之，该项目将把反问题的稀疏正则化提升到一个新的水平。

One fundamental task in many scientific and engineering disciplines is to probe the world around us, often in the form of deducing physical laws or determining the parameters therein from experimental data. This gives rise to a wide variety of inverse problems of inferring the cause of observed physical phenomena or determining medium properties of the concerned object from the measured responses. A model based approach to inverse problems assumes a known relation between the unknown physical quantity and the measured responses, in the form of linear or nonlinear equations. Inverse problems are numerically challenging to solve since they are unstable with respect to data noise. One of the most successful and powerful techniques is to incorporate a priori knowledge by means of regularization. This project focuses on one specific regularization technique based on sparsity constraints.In sparsity regularization, one looks for an approximate solution that has as few nonzero entries (or a small support) as possible. That is, we aim at explaining the physical phenomenon by simply using a small number of parameters. In this project, we study the mathematical theory, computational techniques and practical applications of the approach, especially by using a nonconvex penalty on the sought-for solution. There are several nonconvex penalties proposed in the literature, especially in statistics and machine learning, and we shall consider, for example, the popular l0 penalty, bridge penalty, smoothly clipped absolute deviation, minmax concavity penalty and clipped l1 penalties. Despite their popularity, their use and study in the context of inverse problems remain very limited. The proposed project examines these techniques in the framework of inverse problems theory. Specifically, we focus on the following objectives during the project period: (a) to develop a computational method of primal-dual active set type for efficiently solving the regularized model (with nonconvex penalties) and rigorously establish the convergence of the algorithm; (b) to develop applicable choice rules for the regularization parameter, and to analyze the structural properties of "local" minimizers; (3) to apply the nonconvex approach to tomography imaging, by combining it with an adaptive finite element method. The computational technique to be developed, primal dual active set algorithm, is widely applicable to many other areas, especially machine learning and statistics, since these nonconvex models have their origins and motivations there. Mathematically, the theory of sparsity regularization will shed valuable lights into the analysis of nonsmooth regularization, which shares common structures with many important mathematical models arising in imaging and signal processing. The specific choice rule for the regularization parameter will enable automatic parameter selection yet with rigorous theoretical justification, and improve the efficiency of the current practice based on tedious trial and error. The research in tomography imaging, i.e., the application of nonconvex penalty and adaptive algorithm, will directly impact the medical imaging community. There are a large group of researchers working on tomography imaging that will benefit directly from the research, and the obtained research results will be presented to them continuously. More generally, it provides guidance for developing efficient algorithms for nonlinear inverse problems for differential equations. In summary, the project will take sparsity regularization for inverse problems to the next level.