Equal But Not The Same: Understanding the Implicit Relationship Between Persuasive Images and Text
Equal But Not The Same: Understanding the Implicit Relationship Between Persuasive Images and Text
复制标题
DOI：
--
复制
发表时间：
2018-07
期刊：
影响因子：
--
通讯作者：
Mingda Zhang;R. Hwa;Adriana Kovashka
中科院分区：
文献类型：
--
作者：
Mingda Zhang;R. Hwa;Adriana Kovashka
广告中的图像和文字以复杂的、非文字的方式相互作用。这两个渠道通常是互补的，每个渠道讲述故事的不同部分。目前的方法，如图像字幕方法，只检查字面上的，冗余的关系，其中图像和文本显示完全相同的内容。为了理解更复杂的关系，我们首先收集广告解释的数据集，以确定同一视觉广告中的图像和口号是否形成平行（传达相同的信息，但实际上没有说同样的话）或非平行关系，在亚马逊土耳其机器人招聘的工人的帮助下，我们开发了各种功能，可以捕捉图像的创造力和文本的特殊性或模糊性，以及分析通道内和通道间语义的方法。我们表明，我们的方法优于标准的图像-文本对齐方法预测图像和文本之间的平行/非平行关系。
Images and text in advertisements interact in complex, non-literal ways. The two channels are usually complementary, with each channel telling a different part of the story. Current approaches, such as image captioning methods, only examine literal, redundant relationships, where image and text show exactly the same content. To understand more complex relationships, we first collect a dataset of advertisement interpretations for whether the image and slogan in the same visual advertisement form a parallel (conveying the same message without literally saying the same thing) or non-parallel relationship, with the help of workers recruited on Amazon Mechanical Turk. We develop a variety of features that capture the creativity of images and the specificity or ambiguity of text, as well as methods that analyze the semantics within and across channels. We show that our method outperforms standard image-text alignment approaches on predicting the parallel/non-parallel relationship between image and text.