A Convolutional Attention Network for Extreme Summarization of Source Code
A Convolutional Attention Network for Extreme Summarization of Source Code
复制标题
DOI：
--
复制
发表时间：
2016-02
期刊：
影响因子：
--
通讯作者：
Miltiadis Allamanis;Hao Peng;Charles Sutton
中科院分区：
文献类型：
--
作者：
Miltiadis Allamanis;Hao Peng;Charles Sutton
事实证明，神经网络中的注意机制对于输入和输出没有固定维度的问题很有用。通常，存在本地翻译不变的功能，对于指导模型的注意力很有价值，但是以前的注意体系结构并未构建以专门学习此类功能。我们介绍了一个注意力神经网络，该神经网络采用输入令牌上采用卷积来以上下文依赖性的方式检测本地时间不变和远程局部关注特征。我们将此体系结构应用于将源代码段极端汇总的问题应用于简短的描述性函数名称摘要。使用这些功能，该模型通过对两个注意机制进行边缘化来依次生成摘要：一个基于输入令牌的注意力权重预测下一个摘要令牌，另一个能够将代码令牌直接复制到摘要中。我们证明了我们在10个受欢迎的Java项目上的卷积注意神经网络的表现，表明与以前的注意机制相比，它的性能更好。
Attention mechanisms in neural networks have proved useful for problems in which the input and output do not have fixed dimension. Often there exist features that are locally translation invariant and would be valuable for directing the model's attention, but previous attentional architectures are not constructed to learn such features specifically. We introduce an attentional neural network that employs convolution on the input tokens to detect local time-invariant and long-range topical attention features in a context-dependent way. We apply this architecture to the problem of extreme summarization of source code snippets into short, descriptive function name-like summaries. Using those features, the model sequentially generates a summary by marginalizing over two attention mechanisms: one that predicts the next summary token based on the attention weights of the input tokens and another that is able to copy a code token as-is directly into the summary. We demonstrate our convolutional attention neural network's performance on 10 popular Java projects showing that it achieves better performance compared to previous attentional mechanisms.