Stochastic iterative regularization: theory, algorithms and applications

每当人们寻找观测到的物理现象或观测数据的原因时，就会出现逆问题，例如，从测量中推断出支配定律。这一任务基本上是所有科学发现和技术创新的基础。因此，用于求解逆问题的数学理论和计算技术是核心，例如，在物理学、天文学、医学、工程学和生命科学中，它已经发展成为一个高度跨学科的研究领域。反问题通常是不适定的，在这个意义上，所寻求的解决方案缺乏存在性，唯一性或稳定性的数据扰动。由于噪声是观测数据中固有的，数值算法必须采用专门的技术，通常称为正则化。自A。吉洪诺夫在20世纪60年代，H. Engl et al from 1980 s and many other researchers.该理论在许多研究领域中发挥了重要作用，相关的数值算法也得到了广泛的研究。一个通用的框架是最小化一个目标函数来衡量模型输出和观测数据之间的拟合质量，可能加上一些额外的惩罚项，它涵盖了一大类强大的迭代反演技术，由于数据采集技术的前所未有的进步，大数据集正在成为许多实际反问题的常见场所。医学成像中的突出示例包括计算机断层摄影和光学断层摄影中的动态、多光谱、多能量或多频率数据。不断增加的可用数据量给图像重建带来了巨大的计算挑战，传统的迭代方法可能过于昂贵而无法应用，目前它是从海量数据集中提取有用信息的瓶颈之一。这对于涉及复杂物理模型的问题尤其具有挑战性，其中每个数据集的模拟都非常昂贵。拟议的研究旨在使用机器学习社区内开发的随机迭代技术解决上述突出的计算挑战，并提供相关的理论基础。随机迭代方法的中心思想是，在每一步，只有一个（小）部分的数据集被用来引导迭代的进展，而不是完整的数据集。这允许大幅降低每次迭代的计算成本。这个想法在机器学习社区中受到了极大的关注，尤其是近年来在深度学习方面取得了惊人的成功。实际上，随机梯度下降及其变体是许多深度学习任务背后的主力。该项目的成功完成将通过提供系统的数学和计算框架，包括全面的理论基础，新颖的算法和对具体逆问题的详细研究，大大推进现代图像重建，例如，在医学成像中。

An inverse problem arises whenever one seeks the cause of observed physical phenomena or observational data, e.g., inferring the governing law from the measurements. This task essentially underlies all scientific discoveries and technological innovations. Thus, the mathematical theory and computational techniques for solving inverse problems are central, e.g., in physics, astronomy, medicine, engineering, and life sciences, and it has evolved into a highly interdisciplinary research area. Inverse problems are usually ill-posed in the sense that the sought-for solution lacks existence, uniqueness or stability with respect to data perturbation. Since the noise is inherent in the observational data, the numerical algorithms have to employ specialized techniques, commonly known as regularization. The corresponding mathematical framework in the form of regularization theory is highly developed, since the pioneering works of A. Tikhonov in 1960s, H. Engl et al from 1980s and many other researchers. This theory has played a vital role in many research areas, and related numerical algorithms have also been intensively investigated. One versatile framework is to minimize an objective function measuring the quality of fitting between the model output and observational data, possibly plus some additional penalty term, and it covers a large class of powerful iterative inversion techniques.Due to the unprecedented advances in data acquisition technologies, large datasets are becoming common place for many practical inverse problems. Prominent examples in medical imaging include dynamic, multispectral, multi-energy or multi-frequency data in computed tomography and optical tomography. The ever increasing volume of available data poses enormous computational challenges to image reconstruction, and traditional iterative methods can be too expensive to apply, and currently it represents one of the bottlenecks to extract useful information from the massive dataset. This is especially challenging for problems involving complex physical models, where each data set is very expensive to simulate.The proposed research aims at addressing the aforementioned outstanding computational challenge using stochastic iterative techniques developed within the machine learning community, and providing relevant theoretical underpinnings. The central idea of stochastic iterative methods is that at each step only a (small) portion of the data set is used to steer the progression of the iterates, instead of the full data set. This allows drastically reducing the computational cost per iteration. This idea has received enormous attention within the machine learning community, and especially has achieved stunning success in deep learning in recent years. Actually stochastic gradient descent and its variants are the workhorse behind many deep learning tasks. A successful completion of this project will greatly advance modern image reconstruction by providing a systematic mathematical and computational framework, including comprehensive theoretical underpinnings, novel algorithms and detailed studies on concrete inverse problems, e.g., in medical imaging.