Optical Flow in the Dark
Optical Flow in the Dark
复制标题
DOI：
10.1109/tpami.2021.3130302
复制
发表时间：
2021-11
影响因子：
23.6
通讯作者：
Mingfang Zhang;Yinqiang Zheng;Feng Lu
中科院分区：
文献类型：
--
作者：
Mingfang Zhang;Yinqiang Zheng;Feng Lu
弱光条件下的光流估计是现有方法的一个挑战性任务，并且当前光流数据集缺乏弱光样本。即使在估计之前对暗图像进行增强，这可以实现很好的视觉感知，但仍然会导致次优的光流结果，因为在增强期间可能会破坏运动一致性等信息。我们建议应用一种新的训练策略，直接从新的合成和真实的低光图像中学习光流。具体来说，首先，我们设计了一种方法来收集一个新的光流数据集在多个曝光与共享光流伪标签。然后，我们应用一个两步过程来创建一个合成的低光光流数据集，基于现有的明亮的，通过模拟低光的原始特征，从我们收集的多次曝光的原始图像。为了扩展数据多样性，我们还包括已发布的没有光流标签的低光原始视频。在我们的训练管道中，使用三个数据集，我们创建了两个教师-学生对，以逐步获得所有数据的光流标签。最后，我们将混合训练策略应用于我们的多样化数据集，以生成低光照鲁棒的光流模型。实验结果表明，该方法在图像曝光降低的情况下仍能相对保持光流精度，并在多个实际场景下对不同相机的光流进行了泛化能力测试。
Optical flow estimation in low-light conditions is a challenging task for existing methods and current optical flow datasets lack low-light samples. Even if the dark images are enhanced before estimation, which could achieve great visual perception, it still leads to suboptimal optical flow results because information like motion consistency may be broken during the enhancement. We propose to apply a novel training policy to learn optical flow directly from new synthetic and real low-light images. Specifically, first, we design a method to collect a new optical flow dataset in multiple exposures with shared optical flow pseudo labels. Then we apply a two-step process to create a synthetic low-light optical flow dataset, based on an existing bright one, by simulating low-light raw features from the multi-exposure raw images we collected. To extend the data diversity, we also include published low-light raw videos without optical flow labels. In our training pipeline, with the three datasets, we create two teacher-student pairs to progressively obtain optical flow labels for all data. Finally, we apply a mix-up training policy with our diversified datasets to produce low-light-robust optical flow models for release. The experiments show that our method can relatively maintain the optical flow accuracy as the image exposure descends and the generalization ability of our method is tested with different cameras in multiple practical scenes.