Review Networks for Caption Generation
Review Networks for Caption Generation
复制标题
DOI：
--
复制
发表时间：
2016-05
期刊：
影响因子：
--
通讯作者：
Zhilin Yang;Ye Yuan;Yuexin Wu;William W. Cohen;R. Salakhutdinov
中科院分区：
文献类型：
--
作者：
Zhilin Yang;Ye Yuan;Yuexin Wu;William W. Cohen;R. Salakhutdinov
我们提出了一种新的扩展的编码器-解码器框架，称为审查网络。审查网络是通用的，可以增强任何现有的编码器-解码器模型：在本文中，我们考虑使用CNN和RNN编码器的RNN解码器。回顾网络对编码器隐藏状态执行具有注意力机制的若干回顾步骤，并在每个回顾步骤之后输出思想向量;思想向量用作解码器中的注意力机制的输入。我们表明，传统的编码器-解码器是我们的框架的一个特殊情况。经验上，我们表明，我们的框架提高了国家的最先进的编码器-解码器系统的图像字幕和源代码字幕的任务。
We propose a novel extension of the encoder-decoder framework, called a review network. The review network is generic and can enhance any existing encoder- decoder model: in this paper, we consider RNN decoders with both CNN and RNN encoders. The review network performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a thought vector after each review step; the thought vectors are used as the input of the attention mechanism in the decoder. We show that conventional encoder-decoders are a special case of our framework. Empirically, we show that our framework improves over state-of- the-art encoder-decoder systems on the tasks of image captioning and source code captioning.