EyeTrans: Merging Human and Machine Attention for Neural Code Summarization-猫眼课题宝

DOI：

10.1145/3643732

发表时间：

2024-02

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Yifan Zhang;Jiliang Li;Z. Karas;Aakash Bansal;Toby Jia-Jun Li;Collin McMillan;Kevin Leach;Yu Huang

神经代码摘要利用深度学习模型自动生成代码片段的简短自然语言摘要。Transformer模型的发展导致了在模型设计中广泛使用注意力。虽然现有的工作主要并且几乎完全集中在源代码的静态属性和相关的结构表示（如抽象语法树（AST））上，但很少有研究考虑到人类的注意力，即程序员在检查和理解代码时关注的地方。在本文中，我们开发了一种将人的注意力与机器的注意力相结合的方法来增强神经代码的总结。为了促进这种结合并证明这一假设，我们介绍EyeTrans，它由三个步骤组成：(1)我们进行了广泛的眼动追踪人类研究，以收集和预分析模型训练所需的数据；(2)我们设计了一种以数据为中心的方法，将Transformer架构中的人类注意力与机器注意力集成在一起；(3)我们在两个代码总结任务上进行了全面的实验，以证明将人类注意力整合到Transformer中的有效性。将人的注意力整合在一起，功能总结的性能提高了29.91%，通用代码总结的性能提高了6.39%，这表明了这种组合的巨大好处。通过创建具有挑战性的总结场景，我们进一步探索了在鲁棒性和效率方面的性能，其中EyeTrans展示了有趣的特性。我们还可视化了注意力图，通过合并人类注意力来描述变形金刚中机器注意力的简化效果。这项工作有可能通过引入更多以人为中心的方法和数据来推动软件工程领域的人工智能研究。

Neural code summarization leverages deep learning models to automatically generate brief natural language summaries of code snippets. The development of Transformer models has led to extensive use of attention during model design. While existing work has primarily and almost exclusively focused on static properties of source code and related structural representations like the Abstract Syntax Tree (AST), few studies have considered human attention, that is, where programmers focus while examining and comprehending code. In this paper, we develop a method for incorporating human attention into machine attention to enhance neural code summarization. To facilitate this incorporation and vindicate this hypothesis, we introduce EyeTrans, which consists of three steps: (1) we conduct an extensive eye-tracking human study to collect and pre-analyze data for model training, (2) we devise a data-centric approach to integrate human attention with machine attention in the Transformer architecture, and (3) we conduct comprehensive experiments on two code summarization tasks to demonstrate the effectiveness of incorporating human attention into Transformers. Integrating human attention leads to an improvement of up to 29.91% in Functional Summarization and up to 6.39% in General Code Summarization performance, demonstrating the substantial benefits of this combination. We further explore performance in terms of robustness and efficiency by creating challenging summarization scenarios in which EyeTrans exhibits interesting properties. We also visualize the attention map to depict the simplifying effect of machine attention in the Transformer by incorporating human attention. This work has the potential to propel AI research in software engineering by introducing more human-centered approaches and data.

EyeTrans: Merging Human and Machine Attention for Neural Code Summarization

文献摘要

被引文献

相似文献