Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis
Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis
复制标题
DOI：
10.1175/aies-d-22-0063.1
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Dario Dematties;B. Raut;Seongha Park;Robert C. Jackson;Sean Shahkarami;Yongho Kim;R. Sankaran;P. Beckman;S. Collis;N. Ferrier
中科院分区：
文献类型：
--
作者：
Dario Dematties;B. Raut;Seongha Park;Robert C. Jackson;Sean Shahkarami;Yongho Kim;R. Sankaran;P. Beckman;S. Collis;N. Ferrier
准确的云类型识别和覆盖分析对于了解地球的辐射收支至关重要。传统的计算机视觉方法依赖于云的低级视觉特征来估计云覆盖或天空条件。已经提出了几种手工制作的方法;然而，仍然存在改进的余地。较新的深度神经网络（DNN）在云分割和分类方面表现出了上级性能。然而，这些方法在预处理步骤中需要专家工程干预-在传统方法中-或者在将云或晴空标签分配给像素以训练DNN时需要人工协助。这样的人工调解需要相当多的时间和劳动力成本。我们提出了一种新的自监督学习方法的应用程序，自主提取相关功能，从地面摄像机捕获的天空图像，云的分类和分割。我们评估了一个联合嵌入架构，使用自我知识蒸馏加正则化。我们使用两个数据集来展示网络对天空图像进行分类和分割的能力-一个是从我们的地面摄像机收集的85，000张图像，另一个是来自WSISEG数据库的400张标记图像。我们发现，这种方法可以区分全天空图像的基础上云覆盖，昼夜变化，云底高度。此外，它的语义分割的云区域没有标签。该方法在所有测试任务中显示出具有竞争力的性能，为云表征提供了一种新的选择。
Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ∼ 85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Furthermore, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.