AST-Trans: Code Summarization with Efficient Tree-Structured Attention
AST-Trans: Code Summarization with Efficient Tree-Structured Attention
复制标题
DOI：
10.1145/3510003.3510224
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Ze Tang;Xiaoyu Shen;Chuanyi Li;Jidong Ge;LiGuo Huang;Zheling Zhu;Bin Luo
中科院分区：
文献类型：
--
作者：
Ze Tang;Xiaoyu Shen;Chuanyi Li;Jidong Ge;LiGuo Huang;Zheling Zhu;Bin Luo
代码摘要旨在为源代码生成简短的自然语言描述。最先进的方法遵循基于变换器的编码器-解码器架构。由于源代码是高度结构化的，并遵循严格的语法，其抽象树（AST）被广泛用于编码结构信息。然而，AST比相应的源代码长得多。现有的方法忽略了尺寸约束，并且简单地将整个线性化的AST馈送到编码器中。我们认为，这样一个简单的过程，很难提取真正有用的依赖关系从超长的输入序列。它还带来了显著的计算开销，因为每个节点都需要将自注意力应用于AST中的所有其他节点。为了更有效地编码AST，本文提出了AST-Trans，它利用了AST中的两种节点关系：祖先-后代和兄弟关系。该算法利用树型注意力模型，根据这两种关系，动态地为相关节点分配权重，并排除无关节点。我们进一步提出了一个有效的实现，以支持快速并行计算树结构的注意。在两个代码摘要数据集上，实验结果表明，AST-Trans的性能明显优于最先进的技术，同时比标准变压器的效率高出一倍11所有代码和数据都可以在https://github.com/zetang94/ICSE2022_AST_Trans.git上找到。
Code summarization aims to generate brief natural language descriptions for source codes. The state-of-the-art approaches follow a transformer-based encoder-decoder architecture. As the source code is highly structured and follows strict grammars, its Abstract Syntax Tree (AST) is widely used for encoding structural information. However, ASTs are much longer than the corresponding source code. Existing approaches ignore the size constraint and simply feed the whole linearized AST into the encoders. We argue that such a simple process makes it difficult to extract the truly useful dependency relations from the overlong input sequence. It also incurs significant computational overhead since each node needs to apply self-attention to all other nodes in the AST. To encode the AST more effectively and efficiently, we propose AST-Trans in this paper which exploits two types of node relationships in the AST: ancestor-descendant and sibling relationships. It applies the tree-structured attention to dynamically allocate weights for relevant nodes and exclude irrelevant nodes based on these two relationships. We further propose an efficient implementation to support fast parallel computation for tree-structure attention. On the two code summarization datasets, experimental results show that AST-Trans significantly outperforms the state-of-the-arts while being times more efficient than standard transformers 11All the codes and data are available at https://github.com/zetang94/ICSE2022_AST_Trans.git.