VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning
VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning
复制标题
DOI：
10.1145/3510003.3510162
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Qibin Chen;Jeremy Lacomis;Edward J. Schwartz;Graham Neubig;Bogdan Vasilescu;Claire Le Goues
中科院分区：
文献类型：
--
作者：
Qibin Chen;Jeremy Lacomis;Edward J. Schwartz;Graham Neubig;Bogdan Vasilescu;Claire Le Goues
可变名称对于传达预期的程序行为至关重要。相似性，例如，名字的平均值和平均值是相似的。主要捕获“相关性”（是否完全链接了两个变量），而不是“相似性”（它们实际上具有相同的含义）。这种更严格的感觉。我们观察到这个问题非常适合对比学习不同的输入之间的距离。姓名表示，因此也用于相关的下游任务，例如可变名称相似性搜索或拼写校正。在IDBENCH上胜过最新的，这是一种明确捕获可变相似性的现有基准（与相关性不同）。 - 在替换依赖可变名称的现有或将来的程序分析中使用的变量表示。
Variable names are critical for conveying intended program behavior. Machine learning-based program analysis methods use variable name representations for a wide range of tasks, such as suggesting new variable names and bug detection. Ideally, such methods could capture semantic relationships between names beyond syntactic similarity, e.g., the fact that the names average and mean are similar. Unfortunately, previous work has found that even the best of previous representation approaches primarily capture “relatedness” (whether two variables are linked at all), rather than “similarity” (whether they actually have the same meaning). We propose Varclr, a new approach for learning semantic representations of variable names that effectively captures variable similarity in this stricter sense. We observe that this problem is an excellent fit for contrastive learning, which aims to minimize the distance between explicitly similar inputs, while maximizing the distance between dissimilar inputs. This requires labeled training data, and thus we construct a novel, weakly-supervised variable renaming dataset mined from GitHub edits. We show that Varclr enables the effective application of sophisticated, general-purpose language models like BERT, to variable name representation and thus also to related downstream tasks like variable name similarity search or spelling correction. Varclr produces models that significantly outperform the state-of-the-art on IDBENCH, an existing benchmark that explicitly captures variable similarity (as distinct from relatedness). Finally, we contribute a release of all data, code, and pre-trained models, aiming to provide a drop-in replacement for variable representations used in either existing or future program analyses that rely on variable names.