Deep Learning Model-Aware Regulatization With Applications to Inverse Problems
Deep Learning Model-Aware Regulatization With Applications to Inverse Problems
复制标题
DOI：
10.1109/tsp.2021.3125601
复制
发表时间：
2021-01-01
影响因子：
5.4
通讯作者：
Rodrigues, Miguel R. D.
中科院分区：
文献类型：
--
作者：
Amjad, Jaweria;Lyu, Zhaoyan;Rodrigues, Miguel R. D.
存在各种逆问题-包括医学成像中出现的重建问题-其中人们通常知道将感兴趣的变量映射到观察结果的正向算子。因此，很自然地会问，是否可以在越来越多地用于解决逆问题的深度学习方法中利用正向算子的这种知识。在本文中，我们通过分析深度学习方法对逆问题的泛化误差来提供这样一种方法。特别是，通过建立在算法鲁棒性框架的基础上，我们提供了一个泛化误差界，它封装了与学习问题相关的关键因素，如数据空间的复杂性、训练集的大小、深度神经网络的雅可比矩阵以及前向算子与神经网络的组合的雅可比矩阵。然后，我们提出了一个“即插即用”的正则化器，它利用前向映射的知识来提高网络的泛化能力。我们同样也使用了一种新的方法，使我们能够严格上限的雅可比矩阵的相关运营商，这是更有效的计算比现有的。我们证明了我们的模型感知正则化深度学习算法对其他最先进方法的有效性，这些方法涉及各种子采样算子，例如经典压缩感知任务，图像超分辨率问题和加速磁共振成像（MRI）设置中使用的算子。
There are various inverse problems - including reconstruction problems arising in medical imaging - where one is often aware of the forward operator that maps variables of interest to the observations. It is therefore natural to ask whether such knowledge of the forward operator can be exploited in deep learning approaches increasingly used to solve inverse problems. In this paper, we provide one such way via an analysis of the generalisation error of deep learning approaches to inverse problems. In particular, by building on the algorithmic robustness framework, we offer a generalisation error bound that encapsulates key ingredients associated with the learning problem such as the complexity of the data space, the size of the training set, the Jacobian of the deep neural network and the Jacobian of the composition of the forward operator with the neural network. We then propose a 'plug-and-play' regulariser that leverages the knowledge of the forward map to improve the generalization of the network. We likewise also use a new method allowing us to tightly upper bound the Jacobians of the relevant operators that is much more computationally efficient than existing ones. We demonstrate the efficacy of our model-aware regularised deep learning algorithms against other state-of-the-art approaches on inverse problems involving various sub-sampling operators such as those used in classical compressed sensing tasks, image super-resolution problems and accelerated Magnetic Resonance Imaging (MRI) setups.