Rep2Vec: Repository Embedding via Heterogeneous Graph Adversarial Contrastive Learning
Rep2Vec: Repository Embedding via Heterogeneous Graph Adversarial Contrastive Learning
复制标题
DOI：
10.1145/3534678.3539324
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
Y. Qian;Yiming Zhang;Qianlong Wen;Yanfang Ye;Chuxu Zhang
中科院分区：
文献类型：
--
作者：
Y. Qian;Yiming Zhang;Qianlong Wen;Yanfang Ye;Chuxu Zhang
受软件数量的指数级增长以及基于拉取的开发系统Git的出现推动，大量开源软件在各类社交编码平台上涌现。GitHub作为最大的平台，不仅吸引开发者和研究人员贡献合法软件及与研究相关的源代码，也已成为越来越多网络犯罪分子实施持续网络攻击的热门平台。因此，近来已开发出一些工具，用于学习GitHub上代码库的表征，以应用于各种相关场景（例如恶意代码库检测）。然而，这些工具大多仅关注代码内容，却忽略了代码库之间丰富的关系数据。此外，它们通常需要大量资源来获取足够的标注数据用于模型训练，却忽视了随手可得且有用的未标注数据。为此，我们提出一种新颖的模型Rep2Vec，它整合代码内容、结构关系以及未标注数据来学习代码库的表征。首先，为全面对代码库数据进行建模，我们构建一个代码库异构图（Rep - HG），并通过图神经网络对其进行编码。随后，为充分利用Rep - HG中的未标注数据，我们引入对抗攻击，为对比学习模块生成更具挑战性的对比对，以便同时在节点视角和元路径视角下训练编码器。为减轻编码器应对攻击的工作量，我们进一步设计了一个双流对比学习模块，将对抗图和原始图上的对比学习整合在一起。最后，对预训练的编码器在下游任务中进行微调，并通过知识蒸馏模块进一步增强。在从GitHub收集的数据集上进行的大量实验表明，与最先进的方法相比，Rep2Vec在多种代码库任务中都具有有效性。
Driven by the exponential increase of software and the advent of the pull-based development system Git, a large amount of open-source software has emerged on various social coding platforms. GitHub, as the largest platform, not only attracts developers and researchers to contribute legitimate software and research-related source code but has also become a popular platform for an increasing number of cybercriminals to perform continuous cyberattacks. Hence, some tools have been developed to learn representations of repositories on GitHub for various related applications (e.g., malicious repository detection) recently. However, most of them merely focus on code content while ignoring the rich relational data among repositories. In addition, they usually require a mass of resources to obtain sufficient labeled data for model training while ignoring the usefully handy unlabeled data. To this end, we propose a novel model Rep2Vec which integrates the code content, the structural relations, and the unlabeled data to learn the repository representations. First, to comprehensively model the repository data, we build a repository heterogeneous graph (Rep-HG) which is encoded by a graph neural network. Afterwards, to fully exploit unlabeled data in Rep-HG, we introduce adversarial attacks to generate more challenging contrastive pairs for the contrastive learning module to train the encoder in node view and meta-path view simultaneously. To alleviate the workload of the encoder against attacks, we further design a dual-stream contrastive learning module that integrates contrastive learning on adversarial graph and original graph together. Finally, the pre-trained encoder is fine-tuned to the downstream task, and further enhanced by a knowledge distillation module. Extensive experiments on the collected dataset from GitHub demonstrate the effectiveness of Rep2Vec in comparison with state-of-the-art methods for multiple repository tasks.