TestSage: Regression Test Selection for Large-Scale Web Service Testing
TestSage: Regression Test Selection for Large-Scale Web Service Testing
复制标题
DOI：
10.1109/icst.2019.00052
复制
发表时间：
2019-04
期刊：
影响因子：
--
通讯作者：
Hua Zhong;Lingming Zhang;S. Khurshid
中科院分区：
文献类型：
--
作者：
Hua Zhong;Lingming Zhang;S. Khurshid
回归测试是软件开发中一项重要但昂贵的活动。在各种类型的测试中，Web服务测试通常是最昂贵的（由于网络通信），但在商业软件开发中被广泛采用的测试类型之一。回归测试选择（RTS）旨在通过只运行受代码更改影响的测试来减少需要重新测试的测试数量。虽然在过去的几十年中已经提出了大量的RTS技术，这些技术还没有被采用大规模的Web服务测试。这是因为大多数现有的RTS技术要么需要测试和被测代码之间的直接代码依赖性，要么不能以足够的效率应用于大规模系统。在本文中，我们提出了一种新的RTS技术，TestSage，进行RTS的Web服务测试大规模的商业软件。通过较小的开销，TestSage能够收集测试和被测服务之间的细粒度（功能级别）依赖关系，这些依赖关系并不直接相互依赖。TestSage还成功应用于具有超过一百万个功能的大型复杂系统。我们在Google的大规模后端服务上进行了TestSage的实验。实验结果表明，TestSage在运行所有AEC（分析、执行和收集）阶段时减少了34%的测试时间，在不运行收集阶段时减少了50%的测试时间。TestSage已与Google的内部测试框架集成，并在公司日常运行。
Regression testing is an important but expensive activity in software development. Among various types of tests, web service tests are usually one of the most expensive (due to network communications) but widely adopted types of tests in commercial software development. Regression test selection (RTS) aims to reduce the number of tests which need to be retested by only running tests that are affected by code changes. Although a large number of RTS techniques have been proposed in the past few decades, these techniques have not been adopted on large-scale web service testing. This is because most existing RTS techniques either require direct code dependency between tests and code under test or cannot be applied on large scale systems with enough efficiency. In this paper, we present a novel RTS technique, TestSage, that performs RTS for web service tests on large scale commercial software. With a small overhead, TestSage is able to collect fine grained (function level) dependency between test and service under test that do not directly depend on each other. TestSage has also been successfully applied to large complex systems with over a million functions. We conducted experiments of TestSage on a large scale backend service at Google. Experimental results show that TestSage reduces 34% of testing time when running all AEC (Analysis, Execution and Collection) phases, 50% of testing time while running without collection phase. TestSage has been integrated with internal testing framework at Google and runs day-to-day at the company.