Learning Visual Emotion Representations From Web Data
Learning Visual Emotion Representations From Web Data
复制标题
DOI：
10.1109/cvpr42600.2020.01312
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Zijun Wei;Jianming Zhang;Zhe L. Lin;Joon-Young Lee;Niranjan Balasubramanian;Minh Hoai;D. Samaras
中科院分区：
文献类型：
--
作者：
Zijun Wei;Jianming Zhang;Zhe L. Lin;Joon-Young Lee;Niranjan Balasubramanian;Minh Hoai;D. Samaras
我们提出了一种可扩展的方法来学习强大的视觉特征以进行情感识别。情感识别的一个关键瓶颈是缺乏可用于学习视觉情感特征的大规模数据集。为此，我们策划了一个基于网络的大型数据集 StockEmotion，其中包含超过一百万张图像。 StockEmotion使用690个情感相关标签作为标签，为我们提供了一组细粒度且多样化的情感标签，规避了手动获取情感注释的困难。我们使用该数据集来训练特征提取网络 EmotionNet，并使用联合文本和视觉嵌入以及文本蒸馏进一步规范化该网络。我们的实验结果表明，在 StockEmotion 数据集上训练的 EmotionNet 在四种不同的视觉情感任务上优于 SOTA 模型。我们的联合嵌入训练方法的另一个好处是，EmotionNet 在具有挑战性的视觉情感数据集 EMOTIC 上相对于完全监督的基线实现了有竞争力的零样本识别性能，这进一步突出了所学习的情感特征的普遍性。
We present a scalable approach for learning powerful visual features for emotion recognition. A critical bottleneck in emotion recognition is the lack of large scale datasets that can be used for learning visual emotion features. To this end, we curate a webly derived large scale dataset, StockEmotion, which has more than a million images. StockEmotion uses 690 emotion related tags as labels giving us a fine-grained and diverse set of emotion labels, circumventing the difficulty in manually obtaining emotion annotations. We use this dataset to train a feature extraction network, EmotionNet, which we further regularize using joint text and visual embedding and text distillation. Our experimental results establish that EmotionNet trained on the StockEmotion dataset outperforms SOTA models on four different visual emotion tasks. An aded benefit of our joint embedding training approach is that EmotionNet achieves competitive zero-shot recognition performance against fully supervised baselines on a challenging visual emotion dataset, EMOTIC, which further highlights the generalizability of the learned emotion features.