Decoupled dynamic group equivariant filter for saliency prediction on omnidirectional image
Decoupled dynamic group equivariant filter for saliency prediction on omnidirectional image
复制标题
DOI：
10.1016/j.neucom.2022.09.107
复制
发表时间：
2022-09
期刊：
影响因子：
6
通讯作者：
Dandan Zhu;Kaiwei Zhang;Guokai Zhang;Qiangqiang Zhou;Xiongkuo Min;Guangtao Zhai;Xiaokang Yang
中科院分区：
文献类型：
--
作者：
Dandan Zhu;Kaiwei Zhang;Guokai Zhang;Qiangqiang Zhou;Xiongkuo Min;Guangtao Zhai;Xiaokang Yang
当前基于卷积神经网络（CNN）的显著性预测模型在预测全方位图像（ODI）上的人类注意力方面取得了坚实的进步。然而，这些采用标准卷积的模型有两个主要缺点：内容不可知和计算密集。为了解决这两个缺点，我们提出了一个解耦的动态组等变滤波器（DDGF）。具体而言，受注意力机制的启发，采用轻量级分支估计空间和信道注意力，我们解耦组等变卷积（即p4卷积）到空间和信道动态组等变滤波器。这样的设计不仅使p4卷积滤波器自适应ODI内容，而且大大降低了计算成本。据我们所知，DDGF是第一个应用于显著性预测任务的解耦动态卷积滤波器。同时，我们观察到在ODI显著性预测中，用DDGF代替标准组等变卷积是有效和高效的。实验结果表明，与其他最先进的方法相比，所提出的DDGF可以实现上级性能。此外，我们进行烧蚀实验，以验证建议DDGF的每个组件的有效性。
Current saliency prediction models based on convolutional neural networks (CNNs) achieve solid improvement in predicting human attention on omnidirectional image (ODI). However, these models that employ standard convolution have two main shortcomings: content-agnostic and computation-intensive. To address these two shortcomings, we propose a decoupled dynamic group equivariant filter (DDGF). Specifically, inspired by the attention mechanism that adopts light-weight branches for estimating spatial and channel attention, we decouple group equivariant convolution (i.e.p4 convolution) into spatial and channel dynamic group equivariant filters. Such a design not only makes p4 convolution filter adaptive to ODI content, but also considerably reduces computational cost. To our best knowledge, the DDGF is the first decoupled dynamic convolution filter that applied to the task of saliency prediction. Meanwhile, we observe that it is effective and efficient when replacing standard group equivariant convolution with DDGF in ODI saliency prediction. Experimental results show that the proposed DDGF can achieve superior performance in comparison with other state-of-the-art methods. Additionally, we conduct ablation experiments to verify the effectiveness of each component of the proposed DDGF.