Learning intra-domain style-invariant representation for unsupervised domain adaptation of semantic segmentation
Learning intra-domain style-invariant representation for unsupervised domain adaptation of semantic segmentation
复制标题
DOI：
10.1016/j.patcog.2022.108911
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Zongyao Li;Ren Togo;Takahiro Ogawa;M. Haseyama
中科院分区：
文献类型：
--
作者：
Zongyao Li;Ren Togo;Takahiro Ogawa;M. Haseyama
在本文中，我们的目标是解决语义分割的无监督域自适应（UDA）的问题，并提高UDA的性能与学习域内风格不变表示的新概念。以前的UDA方法集中在减少源域和目标域之间的域间不一致性。然而，由于两个域的数据分布不同，减少域间不一致性并不能保证训练模型在目标域的泛化能力。因此，为了提高UDA的性能，我们在UDA的研究中首次考虑了目标域的域内多样性，旨在训练模型以很好地推广到不同的域内风格。为了实现这一目标，我们提出了一种自集成的方法来学习域内风格不变的表示，我们引入了一个语义感知的多模态图像到图像的翻译模型，以获得图像与多样化的域内风格。我们的方法在两个合成到真实的适应基准上实现了最先进的性能，并且通过进行广泛的实验证明了我们的方法的有效性。
In this paper, we aim to tackle the problem of unsupervised domain adaptation (UDA) of semantic segmentation and improve the UDA performance with a novel conception of learning intra-domain style-invariant representation. Previous UDA methods focused on reducing the inter-domain inconsistency between the source domain and the target domain. However, due to the different data distributions of the two domains, reducing the inter-domain inconsistency cannot ensure the generalization ability of the trained model in the target domain. Therefore, to improve the UDA performance, we take into consideration the intra-domain diversity of the target domain for the first time in studies on UDA and aim to train the model to generalize well to the diverse intra-domain styles. To achieve this, we propose a self-ensembling method to learn the intra-domain style-invariant representation and we introduce a semantic-aware multimodal image-to-image translation model to obtain images with diversified intra-domain styles. Our method achieves state-of-the-art performance on two synthetic-to-real adaptation benchmarks, and we demonstrate the effectiveness of our method by conducting extensive experiments.