Geometric Analysis and Metric Learning of Instruction Embeddings
Geometric Analysis and Metric Learning of Instruction Embeddings
复制标题
DOI：
10.1109/ijcnn55064.2022.9892426
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Sajib Biswas;T. Barao;John Lazzari;Jeret McCoy;Xiuwen Liu;Alexander Kostandarithes
中科院分区：
文献类型：
--
作者：
Sajib Biswas;T. Barao;John Lazzari;Jeret McCoy;Xiuwen Liu;Alexander Kostandarithes
指令嵌入已被证明对软件逆向工程和自动化程序分析至关重要。然而，由于指令的依赖关系复杂且本身具有可变性，使用在自然语言处理中成功的模型进行指令嵌入可能并不有效。在本文中，我们在标记级别和指令族级别对指令嵌入进行几何分析，结果显示出更大的可变性，并导致在内在分析上性能下降。然后我们提议使用度量学习通过三元组损失来改善指令之间的关系。我们在一个大型指令组数据集上的结果显示出显著的改进。我们还通过研究BERT组件以及变压器模块中用于注意力的内积矩阵的特征，对指令嵌入进行了理论分析。论文被接受发表后，代码将公开可用。
Embeddings for instructions have been shown to be essential for software reverse engineering and automated program analysis. However, due to the complexity of dependencies and inherent variability of instructions, instruction embeddings using models that are successful for natural language processing may not be effective. In this paper, we perform geometric analysis of instruction embeddings at the token level and instruction family level, showing much greater variability and leading to degraded performance on intrinsic analyses. Then we propose to use metric learning to improve the relationships among instructions using triplet loss. Our results on a large dataset of instruction groups shows significant improvements. We also provide a theoretical analysis of the instruction embeddings by looking at the BERT components and characteristics of inner-product matrices for attention in the transformer blocks. The code will be available publicly after the paper is accepted for publication.