Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging
Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging
复制标题
DOI：
10.48550/arxiv.2304.04155
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Ruining Deng;C. Cui;Quan Liu;Tianyuan Yao;Lucas W. Remedios;Shunxing Bao;Bennett A. Landman;L. Wheless;Lori A. Coburn;K. Wilson;Yaohong Wang;Shilin Zhao;A. Fogo;Haichun Yang;Yucheng Tang;Yuankai Huo
中科院分区：
文献类型：
--
作者：
Ruining Deng;C. Cui;Quan Liu;Tianyuan Yao;Lucas W. Remedios;Shunxing Bao;Bennett A. Landman;L. Wheless;Lori A. Coburn;K. Wilson;Yaohong Wang;Shilin Zhao;A. Fogo;Haichun Yang;Yucheng Tang;Yuankai Huo
作为图像分割的基础模型，提出了分段任意模型（SAM）。在1100万张授权且尊重隐私的图像上，使用超过10亿个掩模训练了提示分割模型。该模型支持零镜头图像分割与各种分割提示（例如，点，框，蒙版）。这使得SAM对医学图像分析具有吸引力，特别是对于训练数据很少的数字病理学。在本研究中，我们评估了SAM模型在全切片成像（WSI）上具有代表性的分割任务(1)肿瘤分割，（2)非肿瘤组织分割，(3)细胞核分割）上的零射击分割性能。核心结果：结果表明，零射击SAM模型对大型连接对象的分割性能显著。然而，对于密集的实例对象分割，即使在每个图像上有20个提示（点击/框），它也不能始终达到令人满意的性能。我们还总结了数字病理学的局限性：(1)图像分辨率，(2)多重尺度，(3)提示选择，(4)模型微调。在未来，对来自下游病理分割任务的图像进行少量微调可能有助于模型在密集目标分割中获得更好的性能。
The segment anything model (SAM) was released as a foundation model for image segmentation. The promptable segmentation model was trained by over 1 billion masks on 11M licensed and privacy-respecting images. The model supports zero-shot image segmentation with various segmentation prompts (e.g., points, boxes, masks). It makes the SAM attractive for medical image analysis, especially for digital pathology where the training data are rare. In this study, we evaluate the zero-shot segmentation performance of SAM model on representative segmentation tasks on whole slide imaging (WSI), including (1) tumor segmentation, (2) non-tumor tissue segmentation, (3) cell nuclei segmentation. Core Results: The results suggest that the zero-shot SAM model achieves remarkable segmentation performance for large connected objects. However, it does not consistently achieve satisfying performance for dense instance object segmentation, even with 20 prompts (clicks/boxes) on each image. We also summarized the identified limitations for digital pathology: (1) image resolution, (2) multiple scales, (3) prompt selection, and (4) model fine-tuning. In the future, the few-shot fine-tuning with images from downstream pathological segmentation tasks might help the model to achieve better performance in dense object segmentation.