Attention-Guided Network for Semantic Video Segmentation
Attention-Guided Network for Semantic Video Segmentation
复制标题
DOI：
10.1109/access.2019.2943365
复制
发表时间：
2019-09
期刊：
影响因子：
3.9
通讯作者：
Jiangyun Li;Yikai Zhao;J. Fu;Jiajia Wu;Jing Liu
中科院分区：
文献类型：
--
作者：
Jiangyun Li;Yikai Zhao;J. Fu;Jiajia Wu;Jing Liu
深度卷积神经网络（CNN）模型在语义图像分割方面取得了显着的成功。然而，大多数分割模型基于分类网络，倾向于学习图像级特征，并且由于重复的池化和下采样操作而丢失了大量的空间信息，并且基于CNN的方法对输入不鲁棒，因此直接将现有的分割方法应用于语义视频分割将导致一个实例内以及相邻的相同对象的空间不连续和时间不一致的分割预测。 帧，分别。为了应对这一挑战，我们提出了一种注意力引导网络（AGNet）来自适应增强帧间和帧内特征，以实现更精确的分割预测。具体来说，我们在扩张的全卷积网络（FCN）的顶部附加了一个相邻注意模块（AAM）和一个空间注意模块（SAM），分别对时间和空间维度上的特征相关性进行建模。 AAM 有选择地增强相邻帧中相同对象的帧间特征，以实现时间一致的预测。同时，SAM 选择性地聚合一个实例内的帧内特征，以进行空间连续的预测。最后，我们将两个注意力模块的输出相加，以进一步改进特征表示，这有助于同时在时间和空间维度上进行更精确的分割预测。大量实验证明了该方法的有效性，在 CamVid 数据集上获得了 75.22% 的最先进的并集平均交集 (mIoU)。
Remarkable success has been made by deep convolutional neural network (CNN) models in semantic image segmentation. However, most segmentation models are based on classification networks which tend to learn image-level features and lost abundant spatial information due to repeated pooling and downsampling operations, and the CNN-based methods are not robust to inputs, hence directly applying existing segmentation methods to semantic video segmentation will result in spatially inconsecutive and temporally inconsistent segmentation predictions within one instance and of the same objects across adjacent frames, respectively. To tackle this challenge, we propose an Attention-Guided Network (AGNet) to adaptively strengthen inter-frame and intra-frame features for more precise segmentation predictions. Specifically, we append an adjacent attention module (AAM) and a spatial attention module (SAM) on the top of dilated fully convolutional network (FCN), which model the feature correlations in temporal and spatial dimensions, respectively. The AAM selectively enhances the inter-frame features of the same objects across adjacent frames for temporally consistent predictions. Meanwhile, the SAM selectively aggregates the intra-frame features within one instance for spatially consecutive predictions. Finally, we sum the outputs of the two attention modules to further improve feature representations which contribute to more precise segmentation predictions across temporal and spatial dimensions simultaneously. Extensive experiments demonstrate the effectiveness of the proposed method, obtaining state-of-the-art mean intersection of union (mIoU) of 75.22% on CamVid dataset.