Removal of Image Obstacles for Vehicle-mounted Surrounding Monitoring Cameras by Real-time Video Inpainting
Removal of Image Obstacles for Vehicle-mounted Surrounding Monitoring Cameras by Real-time Video Inpainting
复制标题
DOI：
10.1109/cvprw50498.2020.00115
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Yoshihiro Hirohashi;Kenichi Narioka;M. Suganuma;Xing Liu;Y. Tamatsu;Takayuki Okatani
中科院分区：
文献类型：
--
作者：
Yoshihiro Hirohashi;Kenichi Narioka;M. Suganuma;Xing Liu;Y. Tamatsu;Takayuki Okatani
车辆的全景摄像机（SMC）的实际问题之一是由于粘附在其透镜表面上的物质（例如雨滴和泥土）造成的障碍而导致的图像质量的劣化。这种图像退化可以通过计算机视觉领域中已经研究的图像恢复技术来改善。然而，为了辅助驾驶员，恢复图像的真实的时间处理和保真度是必不可少的，这使大多数现有方法不合格。在这项研究中，我们建议采用最近开发的视频修复方法，可以恢复高保真图像在真实的时间。它使用CNN估计光流，并使用它们将当前帧中的遮挡区域与先前帧中的未遮挡区域进行匹配，从而恢复前者。虽然直接应用程序不会导致满意的结果，由于特殊性的SMC视频，我们表明，两个改进，使其有可能获得良好的效果，在实践中是有用的。一个是使用基于模型的流量估计方法来获得用于训练CNN的目标流量，另一个是改进如何使用估计的流量来匹配当前和先前帧。我们进行了实验，使用真实的图像主要是在城市地区的停车位。包括主观评价在内的结果表明了我们方法的有效性。
One of the practical problems with surrounding view cameras (SMCs) of a vehicle is the degradation of image quality due to obstacles by substances adherent to their lens surface, such as raindrops and mud. Such image degradation could be improved by image restoration techniques that have been studied in the field of computer vision. However, to assist the driver, real time processing and fidelity of the recovered image are essential, which disqualifies most of the existing methods. In this study, we propose to adopt a recently developed video-inpainting method that can restore high-fidelity images in real time. It estimates optical flows using a CNN and use them to match occluded regions in the current frame to unoccluded regions in previous frames, restoring the former. Although the direct application does not lead to satisfactory results due to the peculiarities ofthe SMC videos, we show that two improvements make it possible to obtain good results that are useful in practice. One is to use a model-based flow estimation method to obtain target flows for training the CNN, and the other is to improve how the estimated flows are used to match the current and previous frames. We conducted experiments using real images mainly of parking spaces in urban areas. The results, including subjective evaluation, show the effectiveness of our approach.