High-Resolution Image Inpainting Based on Multi-Scale Neural Network
High-Resolution Image Inpainting Based on Multi-Scale Neural Network
复制标题
基于多尺度神经网络的高分辨率图像修复
DOI：
10.3390/electronics8111370
复制
发表时间：
2019-11-01
期刊：
影响因子：
2.9
通讯作者：
Wu, Baolei
中科院分区：
文献类型：
--
作者：
Sun, Tingzhu;Fang, Weidong;Wu, Baolei
尽管近年来基于生成对抗网络（GAN）的图像绘制在精度和速度上取得了很大的突破，但由于内存限制和训练困难，它们只能处理低分辨率的图像。对于高分辨率的图像，所绘制的区域变得模糊，令人不快的边界变得可见。在现有先进的图像生成网络的基础上，提出了一种基于多尺度神经网络的高分辨率图像绘制方法。该方法采用内容重建和纹理细节恢复两阶段网络。在保持视觉上可信的模糊纹理后，我们进一步恢复更精细的细节，以产生更平滑，更清晰，更连贯的涂漆结果。然后提出了一种特殊的图像补图应用场景，即删除图像中多余的行人，保证背景恢复的真实性。它包括行人检测，识别多余的行人，并用看似正确的内容填充他们。为了提高应用场景下图像绘制的精度，我们提出了一种新的掩码数据集，该数据集将COCO数据集中的字符作为掩码。最后，我们在COCO和VOC数据集上对我们的方法进行了评估。实验结果表明，该方法可以产生更清晰、更连贯的图像绘制结果，特别是对于高分辨率图像，并且所提出的掩模数据集在特殊应用场景下可以产生更好的图像绘制效果。
Although image inpainting based on the generated adversarial network (GAN) has made great breakthroughs in accuracy and speed in recent years, they can only process low-resolution images because of memory limitations and difficulty in training. For high-resolution images, the inpainted regions become blurred and the unpleasant boundaries become visible. Based on the current advanced image generation network, we proposed a novel high-resolution image inpainting method based on multi-scale neural network. This method is a two-stage network including content reconstruction and texture detail restoration. After holding the visually believable fuzzy texture, we further restore the finer details to produce a smoother, clearer, and more coherent inpainting result. Then we propose a special application scene of image inpainting, that is, to delete the redundant pedestrians in the image and ensure the reality of background restoration. It involves pedestrian detection, identifying redundant pedestrians and filling in them with the seemingly correct content. To improve the accuracy of image inpainting in the application scene, we proposed a new mask dataset, which collected the characters in COCO dataset as a mask. Finally, we evaluated our method on COCO and VOC dataset. the experimental results show that our method can produce clearer and more coherent inpainting results, especially for high-resolution images, and the proposed mask dataset can produce better inpainting results in the special application scene.