Automatic Image Captioning Based on ResNet50 and LSTM with Soft Attention
Automatic Image Captioning Based on ResNet50 and LSTM with Soft Attention
复制标题
基于 ResNet50 和 LSTM 的自动图像描述，具有软注意力
DOI：
10.1155/2020/8909458
复制
发表时间：
2020-10-21
影响因子：
--
通讯作者：
Wang, Zhengkui
中科院分区：
文献类型：
--
作者：
Chu, Yan;Yue, Xiao;Wang, Zhengkui
为图像自动添加适当的描述已经成为一个有趣且具有挑战性的问题。在本文中，我们提出了一个联合模型AICRL，它能够基于ResNet50和LSTM进行自动图像字幕。AICRL由一个编码器和一个解码器组成。编码器采用基于卷积神经网络的ResNet50，通过将其嵌入到固定长度的向量中来创建给定图像的广泛表示。解码器采用LSTM、递归神经网络和软注意力机制设计，可以选择性地将注意力集中在图像的某些部分，以预测下一句话。我们在MS COCO 2014大数据集上训练了AICRL，以最大限度地提高给定训练图像的目标描述语句的可能性，并在BLEU，METEROR和CIDER等各种指标中对其进行评估。我们的实验结果表明，AICRL是有效的生成图像的字幕。
Captioning the images with proper descriptions automatically has become an interesting and challenging problem. In this paper, we present one joint model AICRL, which is able to conduct the automatic image captioning based on ResNet50 and LSTM with soft attention. AICRL consists of one encoder and one decoder. The encoder adopts ResNet50 based on the convolutional neural network, which creates an extensive representation of the given image by embedding it into a fixed length vector. The decoder is designed with LSTM, a recurrent neural network and a soft attention mechanism, to selectively focus the attention over certain parts of an image to predict the next sentence. We have trained AICRL over a big dataset MS COCO 2014 to maximize the likelihood of the target description sentence given the training images and evaluated it in various metrics like BLEU, METEROR, and CIDEr. Our experimental results indicate that AICRL is effective in generating captions for the images.