News2meme: An Automatic Content Generator from News Based on Word Subspaces from Text and Image
News2meme: An Automatic Content Generator from News Based on Word Subspaces from Text and Image
复制标题
News2meme：基于文本和图像的单词子空间的新闻自动内容生成器
DOI：
10.23919/mva.2019.8757876
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
K. Fukui
中科院分区：
文献类型：
--
作者：
Erica K. Shimomoto;L. S. Souza;B. Gatto;K. Fukui
互联网用户通过使用各种媒体格式进行内容创建。最流行的形式之一是“网络模因”，它通常用一个图像和一句流行语来描述对事件的普遍看法。在本文中，我们提出了news 2 meme，一种从新闻文章中自动生成模因的方法，我们的目标是有效地匹配文字和图像。我们的方法，这是两个多媒体检索问题，相同的输入新闻文本：1）图像检索任务的输出是一个模因图像; 2）文本检索任务的输出是一个流行语。这两个输出被组合以生成新闻文章的模因。我们通过word 2 vec表示将文本和流行语表示为词向量集。为了类似地处理图像，我们使用深度神经网络从图像中提取标签集。然后，这些标签通过word 2 vec被翻译成相同向量空间中的词向量。最后，我们用一个词子空间来表示一组词向量中特征的内在变异性。通过词子空间比较，可以直接比较图像和文本，使跨媒体检索成为可能。进行了初步的实验，以评估我们的框架。
Internet users engage in content creation by using various media formats. One of the most popular forms is the “internet meme”, which often depicts the general opinion about events with an image and a catchphrase. In this paper, we propose news2meme, a method for automatically generating memes from a news article, where we aim to match words and images efficiently. We approach this as two multimedia retrieval problems with the same input news text: 1) An image retrieval task where the output is a meme image; 2) A text retrieval task where the output is a catchphrase. These two outputs are combined to generate the meme for the news article. We represent texts and catchphrases as sets of word vectors through the word2vec representation. To handle images similarly, we extract sets of tags from the images using a deep neural network. These tags are then translated to word vectors in the same vector space through word2vec. Finally, we represent the intrinsic variability of features in a set of word vectors with a word subspace. Through word subspaces comparison, we can directly compare images and texts, making retrieval across media formats possible. A preliminary experiment was performed to evaluate our framework.