H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation
H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation
复制标题
DOI：
10.1109/tmi.2022.3169449
复制
发表时间：
2022-10-01
影响因子：
10.6
通讯作者：
Chen, Danny Z.
中科院分区：
文献类型：
--
作者：
Liang, Peixian;Zhang, Yizhe;Chen, Danny Z.
基于深度学习的语义分割方法在生物医学图像分割中取得了很好的效果，生成了高质量的概率图，可以提取丰富的实例信息，便于进行良好的实例分割。虽然许多人致力于开发新的DL语义切分模型，但对于如何有效地探索其概率图以获得最佳实例切分这一关键问题，人们关注较少。我们观察到，DL语义分割模型的概率图可以用来生成许多可能的实例候选，并且可以通过从其中选择一组优化的候选作为输出实例来实现准确的实例分割。此外，生成的候选实例形成行为良好的分层结构(林)，其允许以优化的方式选择实例。因此，我们提出了一种新的框架，称为分层地球移动距离(H-EMD)，例如在生物医学2D+时间视频和3D图像中的分割，它巧妙地结合了一致的实例选择和语义分割生成的概率图。H-EMD包含两个主要阶段：(1)实例候选生成：通过在森林结构中生成多个实例候选来获取概率图中的实例结构信息；(2)实例候选选择：从候选集合中选择实例用于最终实例分割。将实例候选林上的关键实例选择问题描述为基于推土机距离的优化问题，并用整数线性规划方法进行求解。在8个生物医学视频或3D数据集上的广泛实验表明，H-EMD始终增强了DL语义分割模型，并且与最先进的方法具有很强的竞争力。
Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of "optimized" candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover's distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages: (1) instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure; (2) instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover's distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.