ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media
ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media
复制标题
DOI：
10.1109/cvpr46437.2021.00348
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Z. Shi;Ethan Tseng;Mario Bijelic;W. Ritter;Felix Heide
中科院分区：
文献类型：
--
作者：
Z. Shi;Ethan Tseng;Mario Bijelic;W. Ritter;Felix Heide
恶劣的天气条件，包括雪、雨和雾，对人类和计算机视觉都构成了重大挑战。处理这些环境条件对于安全决策至关重要，特别是在自动驾驶汽车，机器人和无人机中。然而，今天的大多数监督成像和视觉方法都依赖于在真实的世界中收集的训练数据，这些数据偏向于良好的天气条件，浓雾，雪和大雨是这些数据集中的离群值。在没有训练数据的情况下，更不用说配对数据了，现有的自动驾驶汽车通常会将自己限制在良好的条件下，并在检测到浓雾或积雪时停止。在这项工作中，我们通过结合合成和间接监督来解决缺乏监督训练数据的问题。我们提出了ZeroScatter，一种域转移方法，用于将在恶劣天气下拍摄的仅RGB捕获转换为清晰的白天场景。ZeroScatter以一种联合的方式利用基于模型的、时间的、多视图的、多模态的和对抗性的线索，使我们能够在不成对的、有偏见的数据上进行训练。我们对所提出的方法进行了野外捕获评估，在受控雾室测量中，所提出的方法比现有的单目去散射方法的PSNR高出2.8 dB。
Adverse weather conditions, including snow, rain, and fog, pose a major challenge for both human and computer vision. Handling these environmental conditions is essential for safe decision making, especially in autonomous vehicles, robotics, and drones. Most of today’s supervised imaging and vision approaches, however, rely on training data collected in the real world that is biased towards good weather conditions, with dense fog, snow, and heavy rain as outliers in these datasets. Without training data, let alone paired data, existing autonomous vehicles often limit themselves to good conditions and stop when dense fog or snow is detected. In this work, we tackle the lack of supervised training data by combining synthetic and indirect supervision. We present ZeroScatter, a domain transfer method for converting RGB-only captures taken in adverse weather into clear daytime scenes. ZeroScatter exploits model-based, temporal, multi-view, multi-modal, and adversarial cues in a joint fashion, allowing us to train on unpaired, biased data. We assess the proposed method on in-the-wild captures, and the proposed method outperforms existing monocular descattering approaches by 2.8 dB PSNR on controlled fog chamber measurements.