A Framework for Checking Regression Test Selection Tools
A Framework for Checking Regression Test Selection Tools
复制标题
回归测试选择工具的检验框架
DOI：
10.1109/icse.2019.00056
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Chenguang Zhu;Owolabi Legunsen;A. Shi;Miloš Gligorić
中科院分区：
文献类型：
--
作者：
Chenguang Zhu;Owolabi Legunsen;A. Shi;Miloš Gligorić
回归测试选择 (RTS) 通过仅重新运行可能因代码更改而改变行为的测试来降低回归测试成本。研究人员和大型软件组织最近开发并采用了多种 RTS 工具来应对快速增长的回归测试成本。随着 RTS 工具得到广泛采用，检查它们是否正确且高效变得至关重要。不幸的是，检查 RTS 工具目前仅依赖于 RTS 工具开发人员手动编写的有限测试。我们推出 RTSCheck，这是第一个用于检查 RTS 工具的框架。 RTSCheck 将不断发展的程序（即程序修订序列）提供给 RTS 工具，并根据现有 RTS 测试套件启发的规则检查输出。违反这些规则可能是由于 RTS 工具行为与预期存在偏差，并且表明该工具存在错误。 RTSCheck 使用三个组件来获取不断发展的程序：(1) AutoEP 自动生成不断发展的程序和相应的测试，(2) DefectsEP 使用来自 bug 数据库的有缺陷和固定的程序修订，以及 (3) EvoEP 使用来自实际开源项目历史的程序修订序列。我们使用 RTSCheck 检查了三个最近开发的 Java RTS 工具：Clover、Ekstazi 和 STARTS。 RTSCheck 在这三个工具中发现了 27 个错误。
Regression test selection (RTS) reduces regression testing costs by re-running only tests that can change behavior due to code changes. Researchers and large software organizations recently developed and adopted several RTS tools to deal with the rapidly growing costs of regression testing. As RTS tools gain adoption, it becomes critical to check that they are correct and efficient. Unfortunately, checking RTS tools currently relies solely on limited tests that RTS tool developers manually write. We present RTSCheck, the first framework for checking RTS tools. RTSCheck feeds evolving programs (i.e., sequences of program revisions) to an RTS tool and checks the output against rules inspired by existing RTS test suites. Violations of these rules are likely due to deviations from expected RTS tool behavior, and indicative of bugs in the tool. RTSCheck uses three components to obtain evolving programs: (1) AutoEP automatically generates evolving programs and corresponding tests, (2) DefectsEP uses buggy and fixed program revisions from bug databases, and (3) EvoEP uses sequences of program revisions from actual open-source projects' histories. We used RTSCheck to check three recently developed RTS tools for Java: Clover, Ekstazi, and STARTS. RTSCheck discovered 27 bugs in these three tools.