ECSTATIC: An Extensible Framework for Testing and Debugging Configurable Static Analysis
ECSTATIC: An Extensible Framework for Testing and Debugging Configurable Static Analysis
复制标题
DOI：
10.1109/icse48619.2023.00056
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Austin Mordahl;Zenong Zhang;Dakota Soles;Shiyi Wei
中科院分区：
文献类型：
--
作者：
Austin Mordahl;Zenong Zhang;Dakota Soles;Shiyi Wei
测试和调试静态分析的实施是一项具有挑战性的任务，通常涉及在乏味且无关紧要的过程中的域专家手动努力。在这项工作中，我们提出了一种方法，可以极大地改善具有配置选项的静态分析仪的自动化。我们方法的核心是这些选项之间存在的理论部分阶订单关系的新颖调整，以推理以不同配置运行静态分析仪的实际结果的正确性。这允许对静态分析仪进行自动测试，这些分析仪具有明确定义的Oracles，然后进行自动化的Delta调试，即使在未在输入程序上定义地面真相的情况下也是如此。为了将这种方法应用于许多静态分析工具，我们设计和实施了狂喜，易于扩展的开源框架。我们将四种流行的静态分析工具（Soot，Wala，Doop和Flowdroid）整合到了狂喜中。我们的评估表明，运行狂喜的人在四个工具中检测到74个部分订单错误，并产生减少的错误诱导程序以帮助调试。我们报告了42个错误；在我们收到响应的所有情况下，工具开发人员确认报告的工具行为都是意想不到的。到目前为止，已经解决了三个错误，并且正在进行讨论以修复更多内容。
Testing and debugging the implementation of static analysis is a challenging task, often involving significant manual effort from domain experts in a tedious and unprincipled process. In this work, we propose an approach that greatly improves the automation of this process for static analyzers with configuration options. At the core of our approach is the novel adaptation of the theoretical partial order relations that exist between these options to reason about the correctness of actual results from running the static analyzer with different configurations. This allows for automated testing of static analyzers with clearly defined oracles, followed by automated delta debugging, even in cases where ground truths are not defined over the input programs. To apply this approach to many static analysis tools, we design and implement ECSTATIC, an easy-to-extend, open-source framework. We have integrated four popular static analysis tools, SOOT, WALA, DOOP, and FlowDroid, into ECSTATIC. Our evaluation shows running ECSTATIC detects 74 partial order bugs in the four tools and produces reduced bug-inducing programs to assist debugging. We reported 42 bugs; in all cases where we received responses, the tool developers confirmed the reported tool behavior was unintended. So far, three bugs have been fixed and there are ongoing discussions to fix more.