Stochastic iterative regularization: theory, algorithms and applications

每当人们寻找观察到的物理现象或观测数据的原因，例如，从测量结果推断支配规律时，就会出现一个逆问题。这项任务基本上是所有科学发现和技术创新的基础。因此，解决逆问题的数学理论和计算技术是核心，例如在物理学、天文学、医学、工程学和生命科学中，它已经发展成为一个高度交叉的研究领域。反问题通常是不适定的，因为所求的解相对于数据扰动缺乏存在性、唯一性或稳定性。由于噪声是观测数据固有的，所以数值算法必须使用专门的技术，通常称为正则化。自20世纪60年代的A.Tikhonov、80年代的H.Engl等人的开创性工作以来，相应的正则化理论形式的数学框架得到了高度发展。这一理论在许多研究领域起到了至关重要的作用，相关的数值算法也得到了深入的研究。一个通用的框架是最小化一个衡量模型输出和观测数据之间拟合质量的目标函数，可能加上一些额外的惩罚项，它涵盖了一大类强大的迭代反演技术。由于数据采集技术的前所未有的进步，大数据集正成为许多实际反问题的常见场所。医学成像的突出例子包括计算机层析成像和光学层析成像中的动态、多光谱、多能量或多频率数据。不断增长的可用数据量给图像重建带来了巨大的计算挑战，传统的迭代方法应用起来代价太高，目前是从海量数据集中提取有用信息的瓶颈之一。这对于涉及复杂物理模型的问题尤其具有挑战性，其中每个数据集的模拟都非常昂贵。拟议的研究旨在利用机器学习社区内开发的随机迭代技术来解决上述突出的计算挑战，并提供相关的理论基础。随机迭代方法的中心思想是，在每一步，只使用数据集的(一小部分)部分来引导迭代的进行，而不是使用整个数据集。这允许大幅降低每次迭代的计算成本。这一想法在机器学习界得到了极大的关注，特别是近年来在深度学习方面取得了惊人的成功。实际上，随机梯度下降及其变体是许多深度学习任务背后的主力。该项目的成功完成将提供一个系统的数学和计算框架，包括全面的理论基础、新的算法和对具体逆问题的详细研究，例如在医学成像中，从而极大地促进现代图像重建。

An inverse problem arises whenever one seeks the cause of observed physical phenomena or observational data, e.g., inferring the governing law from the measurements. This task essentially underlies all scientific discoveries and technological innovations. Thus, the mathematical theory and computational techniques for solving inverse problems are central, e.g., in physics, astronomy, medicine, engineering, and life sciences, and it has evolved into a highly interdisciplinary research area. Inverse problems are usually ill-posed in the sense that the sought-for solution lacks existence, uniqueness or stability with respect to data perturbation. Since the noise is inherent in the observational data, the numerical algorithms have to employ specialized techniques, commonly known as regularization. The corresponding mathematical framework in the form of regularization theory is highly developed, since the pioneering works of A. Tikhonov in 1960s, H. Engl et al from 1980s and many other researchers. This theory has played a vital role in many research areas, and related numerical algorithms have also been intensively investigated. One versatile framework is to minimize an objective function measuring the quality of fitting between the model output and observational data, possibly plus some additional penalty term, and it covers a large class of powerful iterative inversion techniques.Due to the unprecedented advances in data acquisition technologies, large datasets are becoming common place for many practical inverse problems. Prominent examples in medical imaging include dynamic, multispectral, multi-energy or multi-frequency data in computed tomography and optical tomography. The ever increasing volume of available data poses enormous computational challenges to image reconstruction, and traditional iterative methods can be too expensive to apply, and currently it represents one of the bottlenecks to extract useful information from the massive dataset. This is especially challenging for problems involving complex physical models, where each data set is very expensive to simulate.The proposed research aims at addressing the aforementioned outstanding computational challenge using stochastic iterative techniques developed within the machine learning community, and providing relevant theoretical underpinnings. The central idea of stochastic iterative methods is that at each step only a (small) portion of the data set is used to steer the progression of the iterates, instead of the full data set. This allows drastically reducing the computational cost per iteration. This idea has received enormous attention within the machine learning community, and especially has achieved stunning success in deep learning in recent years. Actually stochastic gradient descent and its variants are the workhorse behind many deep learning tasks. A successful completion of this project will greatly advance modern image reconstruction by providing a systematic mathematical and computational framework, including comprehensive theoretical underpinnings, novel algorithms and detailed studies on concrete inverse problems, e.g., in medical imaging.