3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs.
3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs.
复制标题
DOI：
10.1109/jbhi.2023.3281332
复制
发表时间：
2023-08
影响因子：
7.7
通讯作者：
中科院分区：
文献类型：
--
作者：
未标记成像模式的3D实例分割是一项具有挑战性但必不可少的任务，因为收集专家注释可能是昂贵且耗时的。现有的作品通过部署在不同训练数据上优化的预训练模型或使用两个相对独立的网络顺序进行图像翻译和分割来分割新的模态。在这项工作中，我们提出了一种新的循环分割生成对抗网络（CySGAN），使用具有权重共享的统一网络同时进行图像翻译和实例分割。由于图像翻译层可以在推理时删除，我们提出的模型不会在标准分割模型上引入额外的计算成本。为了优化CySGAN，除了图像翻译的Cycle-GAN损失和注释源域的监督损失外，我们还利用自监督和基于分割的对抗目标，通过利用未标记的目标域图像来增强模型性能。我们基准我们的方法上的任务，3D神经元细胞核分割与注释的电子显微镜（EM）图像和未标记的扩展显微镜（ExM）数据。所提出的CySGAN优于预先训练的通才模型，特征级域自适应模型以及顺序进行图像翻译和分割的基线。我们的实现和新收集的，密集注释的ExM斑马鱼脑核数据集，名为NucExM，可在www.example.com上公开获得。
3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the Cycle-GAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html.