Evaluation of Static Vulnerability Detection Tools With Java Cryptographic API Benchmarks
Evaluation of Static Vulnerability Detection Tools With Java Cryptographic API Benchmarks
复制标题
DOI：
10.1109/tse.2022.3154717
复制
发表时间：
2021-12
影响因子：
7.4
通讯作者：
Sharmin Afrose;Ya Xiao;Sazzadur Rahaman;B. Miller;D. Yao
中科院分区：
文献类型：
--
作者：
Sharmin Afrose;Ya Xiao;Sazzadur Rahaman;B. Miller;D. Yao
几项研究表明，加密API的误用在现实世界的代码中很常见（例如，Apache项目和Android应用程序）。有几个开源和商业安全工具可以自动筛选Java程序以检测滥用。为了比较它们的准确性和安全性保证，我们开发了两个综合基准，名为CryptoAPI-Bench和ApacheCryptoAPI-Bench。CryptoAPI-Bench由181个单元测试用例组成，涵盖基本用例，以及复杂用例，包括过程间，字段敏感，多类测试用例和误用用例的路径敏感数据流。该基准还包括测试假阳性率的正确案例。ApacheCryptoAPI-Bench由来自10个Apache项目的121个加密案例组成。我们评估了四个工具，即SpotBugs，CryptoGuard，CrySL和另一个使用这两个基准测试的工具（匿名）。我们介绍了它们的性能和比较分析。ApacheCryptoAPI-Bench还检查了工具的可扩展性。我们的基准对于在误用检测领域推进最先进的解决方案非常有用。
Several studies showed that misuses of cryptographic APIs are common in real-world code (e.g., Apache projects and Android apps). There exist several open-sourced and commercial security tools that automatically screen Java programs to detect misuses. To compare their accuracy and security guarantees, we develop two comprehensive benchmarks named CryptoAPI-Bench and ApacheCryptoAPI-Bench. CryptoAPI-Bench consists of 181 unit test cases that cover basic cases, as well as complex cases, including interprocedural, field sensitive, multiple class test cases, and path sensitive data flow of misuse cases. The benchmark also includes correct cases for testing false-positive rates. The ApacheCryptoAPI-Bench consists of 121 cryptographic cases from 10 Apache projects. We evaluate four tools, namely, SpotBugs, CryptoGuard, CrySL, and another tool (anonymous) using both benchmarks. We present their performance and comparative analysis. The ApacheCryptoAPI-Bench also examines the scalability of the tools. Our benchmarks are useful for advancing state-of-the-art solutions in the space of misuse detection.