A Saliency Detection and Gram Matrix Transform-Based Convolutional Neural Network for Image Emotion Classification
A Saliency Detection and Gram Matrix Transform-Based Convolutional Neural Network for Image Emotion Classification
复制标题
DOI：
10.1155/2021/6854586
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Zelin Deng;Qiran Zhu;Pei He;Dengyong Zhang;Yuansheng Luo
中科院分区：
文献类型：
--
作者：
Zelin Deng;Qiran Zhu;Pei He;Dengyong Zhang;Yuansheng Luo
利用卷积神经网络(CNN)方法进行图像情感识别是当前深度学习的研究热点。以往的研究倾向于使用从全局视角获得的视觉特征，而忽略了局部视觉特征在情绪唤醒中的作用。此外，CNN的浅层特征地图包含图像内容信息，这种直接从浅层获取的地图描述低层视觉特征可能会导致冗余。为了提高图像情感识别的性能，本文提出了一种改进的CNN。首先，利用显著检测算法定位图像的情感区域，作为辅助信息，更好地进行情感识别。其次，对CNN浅层特征图进行Gram矩阵变换，以降低图像内容信息的冗余度。最后，利用图像情感类别的硬标签和概率标签设计了一种新的损失函数，以减少图像情感主观性的影响。在基准数据集上进行了广泛的实验，包括FI(Flickr和Instagram)、IAPSsubset、ArtPhoto和Abstract。实验结果表明，与现有方法相比，该方法具有较好的应用前景。
Using the convolutional neural network (CNN) method for image emotion recognition is a research hotspot of deep learning. Previous studies tend to use visual features obtained from a global perspective and ignore the role of local visual features in emotional arousal. Moreover, the CNN shallow feature maps contain image content information; such maps obtained from shallow layers directly to describe low-level visual features may lead to redundancy. In order to enhance image emotion recognition performance, an improved CNN is proposed in this work. Firstly, the saliency detection algorithm is used to locate the emotional region of the image, which is served as the supplementary information to conduct emotion recognition better. Secondly, the Gram matrix transform is performed on the CNN shallow feature maps to decrease the redundancy of image content information. Finally, a new loss function is designed by using hard labels and probability labels of image emotion category to reduce the influence of image emotion subjectivity. Extensive experiments have been conducted on benchmark datasets, including FI (Flickr and Instagram), IAPSsubset, ArtPhoto, and Abstract. The experimental results show that compared with the existing approaches, our method has a good application prospect.