Thermal to Visible Image Synthesis Under Atmospheric Turbulence
Thermal to Visible Image Synthesis Under Atmospheric Turbulence
复制标题
DOI：
10.1109/icip46576.2022.9897975
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Kangfu Mei;Yiqun Mei;Vishal M. Patel
中科院分区：
文献类型：
--
作者：
Kangfu Mei;Yiqun Mei;Vishal M. Patel
在许多远程成像的实际应用中，如生物识别和监视，热成像模式通常用于捕获低光和夜间条件下的图像。然而，这种成像系统经常受到大气湍流的影响，这给捕获的图像带来了严重的模糊和变形。这种问题在远程成像中是不可避免的，严重降低了人脸验证的精度。在本文中，我们首先用湍流模拟方法对真实热图像进行了研究。提出了一种基于预训练的StyleGAN2网络，利用自然图像先验直接将热图像转换为可见光谱图像的端到端重建方法。与现有的连续湍流缓解和热图像到可见光图像转换两步方法相比，我们的方法在重建结果的视觉质量和人脸验证精度方面都是有效的。此外，据我们所知，这是第一个研究大气湍流下热图像到可见光图像转换问题的工作。
In many practical applications of long-range imaging such as biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. However, such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. Such an issue is unavoidable in long-range imaging and significantly decreases the face verification accuracy. In this paper, we first investigate the problem with a turbulence simulation method on real-world thermal images. An end-to-end reconstruction method is then proposed which can directly transform thermal images into visible-spectrum images by utilizing natural image priors based on a pre-trained StyleGAN2 network. Compared with the existing two-steps methods of consecutive turbulence mitigation and thermal to visible image translation, our method is demonstrated to be effective in terms of both the visual quality of the reconstructed results and face verification accuracy. Moreover, to the best of our knowledge, this is the first work that studies the problem of thermal to visible image translation under atmospheric turbulence.