Comparing and combining analysis-based and learning-based regression test selection
Comparing and combining analysis-based and learning-based regression test selection
复制标题
比较和结合基于分析和基于学习的回归测试选择
DOI：
10.1145/3524481.3527230
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Shi, August
中科院分区：
文献类型：
--
作者：
Zhang, Jiyang;Liu, Yu;Gligoric, Milos;Legunsen, Owolabi;Shi, August
回归测试——在每个代码版本上重新运行测试以检测新损坏的功能——是重要的并且被广泛实践。但是，由于大量的测试和高频率的代码更改，回归测试是昂贵的。回归测试选择（RTS）通过仅重新运行可能受更改影响的测试子集来优化回归测试。研究人员表明，基于程序分析的RTS可以为（中型）开源项目节省大量的测试时间。从业人员还表明，基于机器学习（ML）的RTS在非常大的代码库（例如Facebook的单存储库）上运行良好。我们将基于分析的RTS和基于ml的RTS结合起来，使用后者来选择前者选择的测试子集。我们首先训练几个新的ML模型，以使用我们通过突变分析获得的训练数据集来学习代码更改对测试结果的影响。然后，我们在10个项目中评估了将ML模型与基于分析的RTS相结合的好处，并与单独使用每种技术进行了比较。将基于ml的RTS与两种基于分析的RTS技术（ekstazi和starts）相结合，分别减少了25.34%和21.44%的测试选择。
Regression testing---rerunning tests on each code version to detect newly-broken functionality---is important and widely practiced. But, regression testing is costly due to the large number of tests and the high frequency of code changes. Regression test selection (RTS) optimizes regression testing by only rerunning a subset of tests that can be affected by changes. Researchers showed that RTS based on program analysis can save substantial testing time for (medium-sized) open-source projects. Practitioners also showed that RTS based on machine learning (ML) works well on very large code repositories, e.g., in Facebook's monorepository. We combine analysis-based RTS and ML-based RTS by using the latter to choose a subset of tests selected by the former. We first train several novel ML models to learn the impact of code changes on test outcomes using a training dataset that we obtain via mutation analysis. Then, we evaluate the benefits of combining ML models with analysis-based RTS on 10 projects, compared with using each technique alone. Combining ML-based RTS with two analysis-based RTS techniques-Ekstazi and STARTS-selects 25.34% and 21.44% fewer tests, respectively.