A visual analysis on recognizability and discriminability of onomatopoeia words with DCNN features
A visual analysis on recognizability and discriminability of onomatopoeia words with DCNN features
复制标题
基于DCNN特征的拟声词识别与判别可视化分析
DOI：
10.1109/icme.2015.7177453
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Keiji Yanai
中科院分区：
文献类型：
--
作者：
Wataru Shimoda;Keiji Yanai
在本文中，我们研究拟声词和图像之间的关系，使用大量的Web图像。本论文的目的是研究日语拟声词所对应的图像是否能被最先进的视觉识别方法识别出来。在我们的工作中，首先，我们收集的图像对应的拟声词使用Web图像搜索引擎，然后我们过滤掉噪声图像，以获得干净的数据集自动图像重新排序方法。接下来，我们使用改进的Fisher向量（IFV）和深度卷积神经网络（DCNN）特征来分析各种象声词图像的可识别性。此外，我们收集对应的名词和拟声词对的图像，我们检查是否与相同的名词和不同的拟声词的图像是视觉上可辨别的或没有。实验结果表明，在用ILSVRC 2013数据预训练的Overfeat网络第7层中提取的DCNN特征具有突出的拟声词图像表征能力，且大多数拟声词具有可识别的视觉特征。
In this paper, we examine the relation between onomatopoeia and images using a large number of Web images. The objective of this paper is to examine if the images corresponding to Japanese onomatopoeia words which express the feeling of visual appearance can be recognized by the state-of-the-art visual recognition methods. In our work, first, we collect the images corresponding to onomatopoeia words using an Web image search engine, and then we filter out noise images to obtain clean dataset with automatic image re-ranking method. Next, we analyze the recognizability of various kinds of onomatopoeia images using improved Fisher vector (IFV) and deep convolutional neural network (DCNN) features. In addition, we collect images corresponding to the pairs of nouns and onomatopoeia words, and we examine if the images associated with the same nouns and the different onomatopoeia words are visually discriminable or not. By the experiments, it has been shown that the DCNN features extracted from the layer 7 of Overfeat's network pre-trained with the ILSVRC 2013 data have prominent ability to represent onomatopoeia images, and most of the onomatopoeia words have visual characteristics which can be recognized.