PalmTree: Learning an Assembly Language Model for Instruction Embedding
PalmTree: Learning an Assembly Language Model for Instruction Embedding
复制标题
DOI：
10.1145/3460120.3484587
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Xuezixiang Li;Qu Yu;Heng Yin
中科院分区：
文献类型：
--
作者：
Xuezixiang Li;Qu Yu;Heng Yin
深度学习已经在众多的二进制分析任务中展示了它的优势，包括函数边界检测、二进制代码搜索、函数原型推理、值集分析等。当将深度学习应用于二进制分析任务时，我们需要决定应该将什么输入输入到神经网络模型中。更具体地说，我们需要回答如何用固定长度的向量表示指令。自动学习指令表示的思想很有趣，但现有的方案未能捕捉到反汇编的独特特征。这些方案忽略了复杂的指令内结构，主要依赖于控制流，其中上下文信息是噪声的，并且会受到编译器优化的影响。在本文中，我们提出了一种称为Palmtree的汇编语言模型，通过在大规模的未标记二进制语料库上进行自我监督训练来生成通用指令嵌入。Palmtree利用三个预训练任务来捕捉汇编语言的各种特征。这些训练任务克服了现有方案中的问题，从而有助于生成高质量的表示。我们进行了内部和外部评估，并将Palmtree与其他指令嵌入方案进行了比较。在内部度量方面，Palmtree具有最好的性能，并且对于所有下游任务，其性能优于其他指令嵌入方案。
Deep learning has demonstrated its strengths in numerous binary analysis tasks, including function boundary detection, binary code search, function prototype inference, value set analysis, etc. When applying deep learning to binary analysis tasks, we need to decide what input should be fed into the neural network model. More specifically, we need to answer how to represent an instruction in a fixed-length vector. The idea of automatically learning instruction representations is intriguing, but the existing schemes fail to capture the unique characteristics of disassembly. These schemes ignore the complex intra-instruction structures and mainly rely on control flow in which the contextual information is noisy and can be influenced by compiler optimizations. In this paper, we propose to pre-train an assembly language model called PalmTree for generating general-purpose instruction embeddings by conducting self-supervised training on large-scale unlabeled binary corpora. PalmTree utilizes three pre-training tasks to capture various characteristics of assembly language. These training tasks overcome the problems in existing schemes, thus can help to generate high-quality representations. We conduct both intrinsic and extrinsic evaluations, and compare PalmTree with other instruction embedding schemes. PalmTree has the best performance for intrinsic metrics, and outperforms the other instruction embedding schemes for all downstream tasks.