Robust deep learning-based multi-image super-resolution using inpainting
Robust deep learning-based multi-image super-resolution using inpainting
复制标题
DOI：
10.1117/1.jei.30.1.013005
复制
发表时间：
2021-01-01
影响因子：
1.1
通讯作者：
Du, Xian
中科院分区：
文献类型：
--
作者：
Yau, Henry;Du, Xian
传统的超分辨技术通常被描述为具有不同优化方法和代价函数的优化问题。即使是对于那些不着边际的案例，这个问题也是病态的。当考虑由于闭塞或缺乏数据而具有未知区域的确定不足的情况时，情况会变得更糟。基于深度学习的方法在解决类似问题方面显示出了希望。最近的一项进展是部分卷积的形式，这种卷积是用来填充图像中的孔洞的。当在适当的深度神经网络中使用时，这种特定的卷积滤波在近似丢失的空间信息方面显示出巨大的前景。所描述的方法是以两个阶段的过程表示的。较低分辨率的图像首先进行配准，然后放置在高分辨率网格上。然后，该问题被处理为一个在画任务，其中使用带有部分卷积滤波的深度神经网络来重建缺失区域。我们将该方法与基于深度学习的单幅图像超分辨率方法和经典的基于两个相似性度量的多幅图像超分辨率方法进行了比较，结果表明，我们的方法对遮挡和配准误差具有更强的鲁棒性，同时也产生了更高质量的输出。？2021年SPIE和IS&T[DOI：10.1117/1.JEI.30.1.013005]
Traditional super-resolution techniques are generally presented as optimization prob-lems with variations in the choice of optimization methods and cost functions. Even for the overdetermined cases, the problem is ill-conditioned. The situation is worsened when consid-ering underdetermined cases with unknown regions due to occlusions or lack of data. Deep learning-based methods have shown promise in solving a similar problem. One recent advance-ment has come in the form of partial convolutions, which were developed to perform infilling of holes in images. When used in an appropriate deep neural network, this particular variant of the convolutional filter has shown great promise in approximating missing spatial information. The method described is formulated as a two-stage process. Lower resolution images are first regis-tered and placed on a high-resolution grid. The problem is then treated as an in-painting task where the missing regions are reconstructed using a deep neural network with partial convolu-tional filters. We compare our method against deep learning-based single image super-resolution methods and classical multi-image super-resolution techniques using two similarity metrics and show that our method is more robust to occlusions and errors in registration while also producing higher quality outputs. ? 2021 SPIE and IS&T [DOI: 10.1117/1.JEI.30.1.013005]