Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network
Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network
复制标题
DOI：
10.1609/aaai.v31i1.10485
复制
发表时间：
2017-02
影响因子：
--
通讯作者：
Jufeng Yang;Ming Sun;Xiaoxiao Sun
中科院分区：
文献类型：
--
作者：
Jufeng Yang;Ming Sun;Xiaoxiao Sun
随着人们越来越倾向于用图像来表达情感，视觉情感分析越来越受到人们的重视。虽然大多数现有的工作都为每个图像分配了单一的主导情绪，但我们通过标签分布学习来解决情绪歧义，这是因为图像通常会唤起多种情绪。提出了两种基于条件概率神经网络(CPNN)的新算法。首先，我们提出了BCPNN，它将图像标签编码成二进制表示，以取代CPNN中使用的无符号整数，并将其作为神经网络的输入。然后，我们通过在地面真值标签上添加噪声和增强情感分布来训练ACPNN模型。由于当前的数据集大多是针对单标签学习进行标注的，因此我们构建了两个新的数据集，其中一个被重新标记在流行的Flickr数据集上，另一个来自Twitter。这些数据集包含20745张带有多个情感标签的图像，这些图像比现有的大十倍以上。实验结果表明，所提出的方法在大规模数据集和其他公开可用的基准上的性能优于最新的工作。
Visual sentiment analysis is raising more and more attention with the increasing tendency to express emotions through images. While most existing works assign a single dominant emotion to each image, we address the sentiment ambiguity by label distribution learning (LDL), which is motivated by the fact that image usually evokes multiple emotions. Two new algorithms are developed based on conditional probability neural network (CPNN). First, we proposed BCPNN which encodes image label into a binary representation to replace the signless integers used in CPNN, and employ it as a part of input for the neural network. Then, we train our ACPNN model by adding noises to ground truth label and augmenting affective distributions. Since current datasets are mostly annotated for single-label learning, we build two new datasets, one of which is relabeled on the popular Flickr dataset and the other is collected from Twitter. These datasets contain 20,745 images with multiple affective labels, which are over ten times larger than the existing ones. Experimental results show that the proposed methods outperform the state-of-the-art works on our large-scale datasets and other publicly available benchmarks.