Performance improvement of weakly supervised fully convolutional networks by skip connections for brain structure segmentation
Performance improvement of weakly supervised fully convolutional networks by skip connections for brain structure segmentation
复制标题
通过大脑结构分割的跳跃连接改进弱监督全卷积网络的性能
DOI：
10.1002/mp.15192
复制
发表时间：
2021
期刊：
影响因子：
3.8
通讯作者：
Kensaku Mori
中科院分区：
文献类型：
--
作者：
Takaaki Sugino;Holger R. Roth;Masahiro Oda;Taichi Kin;Nobuhito Saito;Yoshikazu Nakajima;Kensaku Mori
目的为了神经外科的规划和导航，我们提出了一种基于全卷积网络(FCN)的磁共振图像脑结构分割方法。FCN的能力取决于训练数据(即原始数据和注释数据)和网络结构的质量。注释质量的提高是一个重要的问题，因为它需要大量的人工来标记器官区域。为了解决这个问题，我们重点研究了跳跃连接结构，并利用稀疏注释的脑图像来揭示哪些跳跃连接对于训练FCN是有效的。第一种是具有水平跳跃连接的U-Net结构，它将相同比例的特征地图从编码器传输到解码器。第二种是具有密集卷积层和密集水平跳跃连接的U-Net++体系结构。第三种是具有垂直跳跃连接的全分辨率残差网络(FRRN)结构，该结构在每个下采样尺度路径和全分辨率尺度路径之间传递特征映射。最后一种是水平和垂直跳跃连接相结合的混合体系结构。结果对于稀疏标注MR图像中的大脑、小脑、脑干和血管的多类分割，我们对U-Net、U-Net++、FRN和混合四种FCN方法的分割性能进行了对比评估。实验结果表明，U-Net结构中的水平跳跃连接对于较大尺寸目标的分割是有效的，而FRRN结构中的垂直跳跃连接则改善了较小尺寸目标的分割。在四种FCN结构中，水平和垂直跳跃连接的混合结构取得了最好的效果。然后，我们进行了消融研究，以探索FRRN结构中的哪些跳跃连接有助于改善血管分割。在消融研究中，我们比较了水平路径(HP)、HP和垂直上行路径(HP+VUPS)、HP和垂直下行路径(HP+VDPS)以及HP和垂直上行和下行路径(FRRN)架构之间的分割性能。实验结果表明，垂直向上路径能够有效地提高较小尺寸物体的分割效果。结论从稀疏标注出发，研究了哪种跳跃连接结构在多类脑分割中的有效性。因此，使用垂直跳过连接和水平跳过连接允许FCN提高分段性能。
PurposeFor the planning and navigation of neurosurgery, we have developed a fully convolutional network (FCN)‐based method for brain structure segmentation on magnetic resonance (MR) images. The capability of an FCN depends on the quality of the training data (i.e., raw data and annotation data) and network architectures. The improvement of annotation quality is a significant concern because it requires much labor for labeling organ regions. To address this problem, we focus on skip connection architectures and reveal which skip connections are effective for training FCNs using sparsely annotated brain images.MethodsWe tested 2D FCN architectures with four different types of skip connections. The first was a U‐Net architecture with horizontal skip connections that transfer feature maps at the same scale from the encoder to the decoder. The second was a U‐Net++ architecture with dense convolution layers and dense horizontal skip connections. The third was a full‐resolution residual network (FRRN) architecture with vertical skip connections that pass feature maps between each downsampled scale path and the full‐resolution scale path. The last one was a hybrid architecture with a combination of horizontal and vertical skip connections. We validated the effect of skip connections on medical image segmentation from sparse annotation based on these four FCN architectures, which were trained under the same conditions.ResultsFor multiclass segmentation of the cerebrum, cerebellum, brainstem, and blood vessels from sparsely annotated MR images, we performed a comparative evaluation of segmentation performance among the above four FCN approaches: U‐Net, U‐Net++, FRRN, and hybrid architectures. The experimental results show that the horizontal skip connections in the U‐Net architectures were effective for the segmentation of larger sized objects, whereas the vertical skip connections in the FRRN architecture improved the segmentation of smaller sized objects. The hybrid architecture with both horizontal and vertical skip connections achieved the best results of the four FCN architectures. We then performed an ablation study to explore which skip connections in the FRRN architecture contributed to the improved segmentation of blood vessels. In the ablation study, we compared the segmentation performance between architectures with a horizontal path (HP), an HP and vertical up paths (HP+VUPs), an HP and vertical down paths (HP+VDPs), and an HP and vertical up and down paths (FRRN). We found that the vertical up paths were effective in improving the segmentation of smaller sized objects.ConclusionsThis paper investigated which skip connection architectures were effective for multiclass brain segmentation from sparse annotation. Consequently, using vertical skip connections with horizontal skip connections allowed FCNs to improve segmentation performance.