Learning in the Frequency Domain
Learning in the Frequency Domain
复制标题
DOI：
10.1109/cvpr42600.2020.00181
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Kai Xu;Minghai Qin;Fei Sun;Yuhao Wang;Yen-kuang Chen;Fengbo Ren
中科院分区：
文献类型：
--
作者：
Kai Xu;Minghai Qin;Fei Sun;Yuhao Wang;Yen-kuang Chen;Fengbo Ren
深度神经网络在计算机视觉任务中取得了显著的成功。现有的神经网络主要在具有固定输入大小的空间域中运行。在实际应用中，图像通常很大，必须将其下采样到预定的神经网络输入大小。尽管下采样操作减少了计算量和所需的通信带宽，但它忽略了冗余和显著信息，这导致了精度的下降。受数字信号处理理论的启发，我们从频率角度分析了频谱偏差，提出了一种基于学习的频率选择方法来识别可以在不损失精度的情况下去除的微小频率分量。所提出的频域学习方法利用了众所周知的神经网络的相同结构，例如ResNet-50、MobileNetV2和MASK R-CNN，同时接受频域信息作为输入。实验结果表明，带静态通道选择的频域学习方法比传统的空间下采样方法具有更高的精度，同时进一步减小了输入数据量。具体地，对于相同输入规模的ImageNet分类，该方法在ResNet-50和MobileNetV2上分别获得了1.60%和0.63%的TOP-1准确率改进。即使在输入大小减半的情况下，该方法在ResNet-50上的TOP-1准确率仍提高了1.42%。此外，我们观察到MASK R-CNN在COCO数据集上的平均分割精度提高了0.8%。
Deep neural networks have achieved remarkable success in computer vision tasks. Existing neural networks mainly operate in the spatial domain with fixed input sizes. For practical applications, images are usually large and have to be downsampled to the predetermined input size of neural networks. Even though the downsampling operations reduce computation and the required communication bandwidth, it removes both redundant and salient information obliviously, which results in accuracy degradation. Inspired by digital signal processing theories, we analyze the spectral bias from the frequency perspective and propose a learning-based frequency selection method to identify the trivial frequency components which can be removed without accuracy loss. The proposed method of learning in the frequency domain leverages identical structures of the well-known neural networks, such as ResNet-50, MobileNetV2, and Mask R-CNN, while accepting the frequency-domain information as the input. Experiment results show that learning in the frequency domain with static channel selection can achieve higher accuracy than the conventional spatial downsampling approach and meanwhile further reduce the input data size. Specifically for ImageNet classification with the same input size, the proposed method achieves 1.60% and 0.63% top-1 accuracy improvements on ResNet-50 and MobileNetV2, respectively. Even with half input size, the proposed method still improves the top-1 accuracy on ResNet-50 by 1.42%. In addition, we observe a 0.8% average precision improvement on Mask R-CNN for instance segmentation on the COCO dataset.