Superresolution Image Reconstruction: Selective milestones and open problems
Superresolution Image Reconstruction: Selective milestones and open problems
复制标题
DOI：
10.1109/msp.2023.3271438
复制
发表时间：
2023-07
影响因子：
14.9
通讯作者：
Xin Li;W. Dong;Jinjian Wu;Leida Li;Guangming Shi
中科院分区：
文献类型：
--
作者：
Xin Li;W. Dong;Jinjian Wu;Leida Li;Guangming Shi
在图像和视频等多维信号处理中，超分辨率成像是一个经典问题。在过去的25年里，学术界和工业界一直对从低分辨率（LR）图像中重建高分辨率（HR）图像感兴趣。在这篇教程中，我们基于与先验知识或正则化方法相关的关键见解的演变，回顾了SR技术的发展，从分析表示到数据驱动的深度模型。SR与其他技术领域的共同进化，如自回归建模、稀疏编码和深度学习，将在基于模型和基于学习的方法中得到强调。基于模型的SR包括几何驱动先验、稀疏先验和梯度剖面先验；基于学习的SR涵盖了三种类型的神经网络（NN）架构，即残差网络（ResNet）、生成对抗网络（gan）和预训练模型（PTMs）。基于模型的SR和基于学习的SR从模型-数据不匹配的角度强调了它们的局限性，从而将两者结合起来。我们的新视角允许我们对当前的实践保持健康的怀疑态度，并倡导一种结合基于模型和基于学习的SR优势的混合方法。我们还将讨论几个开放的挑战，包括任意比率、基于参考和特定领域的SR。
In multidimensional signal processing, such as image and video processing, superresolution (SR) imaging is a classical problem. Over the past 25 years, academia and industry have been interested in reconstructing high-resolution (HR) images from their low-resolution (LR) counterparts. We review the development of SR technology in this tutorial article based on the evolution of key insights associated with the prior knowledge or regularization method from analytical representations to data-driven deep models. The coevolution of SR with other technical fields, such as autoregressive modeling, sparse coding, and deep learning, will be highlighted in both model-based and learning-based approaches. Model-based SR includes geometry-driven, sparsity-based, and gradient-profile priors; learning-based SR covers three types of neural network (NN) architectures, namely residual networks (ResNet), generative adversarial networks (GANs), and pretrained models (PTMs). Both model-based and learning-based SR are united by highlighting their limitations from the perspective of model-data mismatch. Our new perspective allows us to maintain a healthy skepticism about current practice and advocate for a hybrid approach that combines the strengths of model-based and learning-based SR. We will also discuss several open challenges, including arbitrary-ratio, reference-based, and domain-specific SR.