All-in-SAM: from Weak Annotation to Pixel-wise Nuclei Segmentation with Prompt-based Finetuning
All-in-SAM: from Weak Annotation to Pixel-wise Nuclei Segmentation with Prompt-based Finetuning
复制标题
DOI：
10.1088/1742-6596/2722/1/012012
复制
发表时间：
2023-07
期刊：
影响因子：
--
通讯作者：
C. Cui;Ruining Deng;Quan Liu;Tianyuan Yao;Shunxing Bao;Lucas W. Remedios;Yucheng Tang;Yuankai Hu
中科院分区：
文献类型：
--
作者：
C. Cui;Ruining Deng;Quan Liu;Tianyuan Yao;Shunxing Bao;Lucas W. Remedios;Yucheng Tang;Yuankai Hu
Segment Anything Model（SAM）是最近提出的一种通用的零拍分割方法中的基于时间序列的分割模型。凭借零触发分割能力，SAM在各种分割任务上实现了令人印象深刻的灵活性和精度。然而，目前的流水线在推理阶段需要手动提示，这对于生物医学图像分割来说仍然是资源密集型的。在本文中，我们引入了一个流水线，该流水线在整个AI开发工作流程（从注释生成到模型微调）中使用SAM，而不是在推理阶段使用提示，而无需在推理阶段手动提示。具体地，SAM首先用于从弱提示（例如，点、边界框）。然后，使用像素级注释来微调SAM分割模型，而不是从头开始训练。我们的实验结果揭示了两个关键发现：1）在公共Monuseg数据集上的核分割任务中，所提出的流水线超越了最先进的方法，以及2）与使用强像素注释数据相比，利用弱注释和少量注释进行SAM微调实现了有竞争力的性能。
The Segment Anything Model (SAM) is a recently proposed prompt-based segmentation model in a generic zero-shot segmentation approach. With the zero-shot segmentation capacity, SAM achieved impressive flexibility and precision on various segmentation tasks. However, the current pipeline requires manual prompts during the inference stage, which is still resource intensive for biomedical image segmentation. In this paper, instead of using prompts during the inference stage, we introduce a pipeline that utilizes the SAM, called all-in-SAM, through the entire AI development workflow (from annotation generation to model finetuning) without requiring manual prompts during the inference stage. Specifically, SAM is first employed to generate pixel-level annotations from weak prompts (e.g., points, bounding box). Then, the pixel-level annotations are used to finetune the SAM segmentation model rather than training from scratch. Our experimental results reveal two key findings: 1) the proposed pipeline surpasses the state-of-the-art methods in a nuclei segmentation task on the public Monuseg dataset, and 2) the utilization of weak and few annotations for SAM finetuning achieves competitive performance compared to using strong pixelwise annotated data.