DesmokeNet: A Two-Stage Smoke Removal Pipeline Based on Self-Attentive Feature Consensus and Multi-Level Contrastive Regularization
DesmokeNet: A Two-Stage Smoke Removal Pipeline Based on Self-Attentive Feature Consensus and Multi-Level Contrastive Regularization
复制标题
DesmokeNet：基于自注意力特征共识和多级对比正则化的两阶段除烟管道
DOI：
10.1109/tcsvt.2021.3106198
复制
发表时间：
2021
影响因子：
8.4
通讯作者：
Sy
中科院分区：
文献类型：
--
作者：
Wei;Hao;H. Fang;I;Yi;Jian;Sy
在图像处理中，烟雾可能会降低可见度并降低高级视觉应用的性能。因此，单图像除烟对于计算机视觉至关重要。目前，现有的除烟算法主要利用手工先验。此外，由于烟雾和霾之间的相似性，这些方法通常采用除霾方法来进行烟雾去除。然而，这些方法不能充分解决浓烟的降解问题，并且可能由于烟雾的非全局和不均匀分布而遭受残留烟雾和颜色失真问题。在本文中，为了解决上述问题，提出了一种称为 DesmokeNet 的端到端深度神经网络。我们建造了两级回收管道来去除不同厚度的烟雾。淡烟和浓烟首先由除烟网络（SRN）局部去除。然后，像素补偿网络（PCN）恢复浓烟中丢失的像素。此外，我们提出了厚度感知像素损失和暗通道损失来抑制残留烟雾。为了进一步提高DesmokeNet的判别能力，我们提出了自注意力特征一致性损失和多级对比正则化损失来提高除烟性能。最后，为了训练所提出的方法，我们构建了第一个包含合成数据和真实数据的大型数据集。大量的实验表明，所提出的方法在数量和质量上都优于其他最先进的方法。
In image processing, smoke may degrade visibility and deteriorate the performance of high-level vision applications. Therefore, single image smoke removal is crucial for computer vision. Currently, existing smoke removal algorithms mainly leverage handcrafted priors. Moreover, these methods usually apply haze removal methods to perform smoke removal due to the similarity between smoke and haze. However, these methods cannot sufficiently address the degradation of thick smoke and may suffer from residual smoke and color distortion problems due to the non-global and non-homogeneous distribution of smoke. In this paper, to solve the aforementioned problems, an end-to-end deep neural network called DesmokeNet is proposed. We construct a two-stage recovered pipeline to remove the smoke in different thicknesses. The light and thick smoke is first removed locally by the smoke removal network (SRN). The missing pixels in the thick smoke are then recovered by the pixel compensation network (PCN). Moreover, we proposed the thickness-aware pixel loss and the dark channel loss to suppress the residual smoke. To further increase the discriminative ability of the DesmokeNet, we proposed self-attentive feature consensus loss and multi-level contrastive regularization loss to improve the performance of smoke removal. Finally, to train the proposed method, we construct the first large-scale dataset containing synthetic and real-world data. Extensive experiments show that the proposed method outperforms favorably against other state-of-the-art methods quantitatively and qualitatively.