ALGES: Active Learning with Gradient Embeddings for Semantic Segmentation of Laparoscopic Surgical Images
ALGES: Active Learning with Gradient Embeddings for Semantic Segmentation of Laparoscopic Surgical Images
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Josiah Aklilu;Serena Yeung
中科院分区：
文献类型：
--
作者：
Josiah Aklilu;Serena Yeung
为了训练计算机视觉模型而对医学图像进行注释是一项非常艰巨的任务，需要花费专家临床医生的时间和资源。主动学习（AL）是一种机器学习范式，它通过故意提出应该标记的数据点来缓解这个问题，以最大限度地提高模型性能。我们提出了一种新的AL分割算法，ALGES，利用梯度嵌入，有效地选择腹腔镜图像标记的一些外部的甲骨文，同时减少注释工作。给定任何未标记的图像，我们的算法将预测的分割视为真相，并计算相对于分割网络中最后一层模型参数的梯度。这些每像素梯度向量的范数对应于模型参数中引起的变化的幅度，并且包含关于模型的预测不确定性的丰富信息。然后，我们的算法以两种方式计算梯度嵌入，我们采用中心查找算法与这些嵌入，以获得代表性和不同的批次在每一轮AL。我们的方法的一个优点是可扩展性的任何模型架构和可区分的损失方案的语义分割。我们将我们的方法应用于腹腔镜胆囊切除术图像的公共数据集，并表明它在选择最具信息量的数据点以改进分割模型方面优于当前的AL算法。我们的代码可在https://github.com/josaklil-ai/surg-active-learning上获得。深度学习模型通过梯度更新进行训练，而引起较大梯度更新的数据点是模型应该学习的数据点。因此，可以通过计算分割模型的权重的梯度并将这些梯度表示为图像级嵌入来捕获图像的不确定性。我们还表明，在语义层面上，考虑引起模型参数大幅更新的特定语义类的像素，使我们能够深入了解图像特定区域的模型不确定性。我们通过实证研究证实了这种直觉，但我们认识到，这种参考框架可以为理解深度学习模型的不确定性提供有意义的见解。
Annotating medical images for the purposes of training computer vision models is an ex-tremely laborious task that takes time and resources away from expert clinicians. Active learning (AL) is a machine learning paradigm that mitigates this problem by deliberately proposing data points that should be labeled in order to maximize model performance. We propose a novel AL algorithm for segmentation, ALGES, that utilizes gradient embeddings to effectively select laparoscopic images to be labeled by some external oracle while reducing annotation effort. Given any unlabeled image, our algorithm treats predicted segmenta-tions as truth and computes gradients with respect to the model parameters of the last layer in a segmentation network. The norms of these per-pixel gradient vectors correspond to the magnitude of the induced change in model parameters and contain rich information about the model’s predictive uncertainty. Our algorithm then computes gradients embeddings in two ways, and we employ a center-finding algorithm with these embeddings to procure representative and diverse batches in each round of AL. An advantage of our approach is extensibility to any model architecture and differentiable loss scheme for semantic segmentation. We apply our approach to a public data set of laparoscopic cholecystectomy images and show that it outperforms current AL algorithms in selecting the most informative data points for improving the segmentation model. Our code is available at https://github.com/josaklil-ai/surg-active-learning . encapsulate uncertainty information while providing a way to ensure diversity in acquired batches during AL. Deep learning models are trained via gradient updates, and data points that bring about large gradient updates are data points that the model should learn. Thus, uncertainty for an image can be captured by computing gradients of the weights for segmentation models, and representing these gradients as an image-level embedding. We have also shown that at the semantic level, considering pixels of a particular semantic class that induce large updates to the model parameters gives us insight into model uncertainty for particular regions of an image. We substantiate this intuition with empirical studies, but we recognize that this frame of reference can provide meaningful insight in understanding uncertainty for deep learning models.