Interactive metamorphic testing of debuggers
Interactive metamorphic testing of debuggers
复制标题
DOI：
10.1145/3293882.3330567
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
S. Tolksdorf;Daniel Lehmann;Michael Pradel
中科院分区：
文献类型：
--
作者：
S. Tolksdorf;Daniel Lehmann;Michael Pradel
在改进代码时，开发人员经常求助于交互式调试器。这些工具的正确性是至关重要的，因为调试器本身的错误可能会误导开发人员，例如，相信执行的代码永远不会到达，或者一个变量在实际执行中有另一个值。然而，调试器很难测试，因为它们的输入由源代码和一系列调试操作（如设置断点或单步执行代码）组成。本文提出了第一种面向调试器的变形测试方法。关键思想是转换调试代码和调试操作，使原始输入和转换后的输入的行为仅在特定方面有所不同。例如，添加断点不应更改被调试程序的控制流。为了支持调试器的交互性，我们引入了交互式变形测试。它与传统的变形测试不同，它确定输入转换和预期的行为变化，它引起的程序在测试运行。我们的评估将该方法应用于Chromium浏览器中广泛使用的调试器，在那里它发现了8个以前未知的错误，真阳性率为51%。所有的bug都已经被开发人员确认，其中一个bug甚至被标记为发布阻塞。
When improving their code, developers often turn to interactive debuggers. The correctness of these tools is crucial, because bugs in the debugger itself may mislead a developer, e.g., to believe that executed code is never reached or that a variable has another value than in the actual execution. Yet, debuggers are difficult to test because their input consists of both source code and a sequence of debugging actions, such as setting breakpoints or stepping through code. This paper presents the first metamorphic testing approach for debuggers. The key idea is to transform both the debugged code and the debugging actions in such a way that the behavior of the original and the transformed inputs should differ only in specific ways. For example, adding a breakpoint should not change the control flow of the debugged program. To support the interactive nature of debuggers, we introduce interactive metamorphic testing. It differs from traditional metamorphic testing by determining the input transformation and the expected behavioral change it causes while the program under test is running. Our evaluation applies the approach to the widely used debugger in the Chromium browser, where it finds eight previously unknown bugs with a true positive rate of 51%. All bugs have been confirmed by the developers, and one bug has even been marked as release-blocking.