Compiler Fuzzing: How Much Does It Matter?
Compiler Fuzzing: How Much Does It Matter?
复制标题
DOI：
10.1145/3360581
复制
发表时间：
2019-10-01
影响因子：
1.8
通讯作者：
Cadar, Cristian
中科院分区：
文献类型：
--
作者：
Marcozzi, Michael;Tang, Qiyi;Cadar, Cristian
尽管最近对编译器的随机测试(模糊)很感兴趣，但几乎没有人评估Fuzzer发现的编译器错误对现实世界应用程序的实际影响。我们首次对一个成熟的编译器中的错误编译错误的有形影响进行了定量和定性研究。我们遵循一种严格的方法，根据(1)错误是否在编译期间触发；(2)生成的汇编代码由于错误的触发而在语法上发生变化的程度；以及(3)这种变化是否会导致回归测试套件失败，或者我们是否可以手动找到因此类变化而触发执行分歧的应用程序输入，来评估错误对已编译应用程序的影响。这项研究是关于从309个Debian包编译1000多万行C/C++代码进行的，使用了Clang/LLVM编译器中四个最先进的Fuzzer发现的12%的历史错误编译错误和现在修复的错误编译错误，以及人类用户编译真实代码或作为正式验证工作的副产品发现的18个错误。结果显示，Fuzzer发现的几乎一半的错误传播到至少一个包的生成的二进制文件中，在这种情况下，通常只有很小一部分二进制文件受到影响，但在运行所有受影响包的测试套件时会导致两次失败。用户报告的和正式的验证错误不会表现出更高的影响，触发错误和一次测试失败的比率较低。手动分析由程序包汇编代码中的一些错误(模糊发现的和非模糊发现的)引起的语法更改的选择表明，这些更改要么没有语义影响，要么它们需要非常特定的运行时环境来触发执行分歧。
Despite much recent interest in randomised testing (fuzzing) of compilers, the practical impact of fuzzer-found compiler bugs on real-world applications has barely been assessed. We present the first quantitative and qualitative study of the tangible impact of miscompilation bugs in a mature compiler. We follow a rigorous methodology where the bug impact over the compiled application is evaluated based on (1) whether the bug appears to trigger during compilation; (2) the extent to which generated assembly code changes syntactically due to triggering of the bug; and (3) whether such changes cause regression test suite failures, or whether we can manually find application inputs that trigger execution divergence due to such changes. The study is conducted with respect to the compilation of more than 10 million lines of C/C++ code from 309 Debian packages, using 12% of the historical and now fixed miscompilation bugs found by four state-of-the-art fuzzers in the Clang/LLVM compiler, as well as 18 bugs found by human users compiling real code or as a by-product of formal verification efforts. The results show that almost half of the fuzzer-found bugs propagate to the generated binaries for at least one package, in which case only a very small part of the binary is typically affected, yet causing two failures when running the test suites of all the impacted packages. User-reported and formal verification bugs do not exhibit a higher impact, with a lower rate of triggered bugs and one test failure. The manual analysis of a selection of the syntactic changes caused by some of our bugs (fuzzer-found and non fuzzer-found) in package assembly code, shows that either these changes have no semantic impact or that they would require very specific runtime circumstances to trigger execution divergence.