SHF: Small:Testing in the Presence of Continuous Change

点击翻译按钮获取中文摘要

Many organizations that develop software are utilizing continuous integrationprocesses, in which engineers merge code frequently with the mainline codebase,and continuously regression test it. The pace at which evolution occurs insuch environments, and the numbers of tests that must be run, can be staggering.To ensure the quality of software in these situations, testing processes needto adapt quickly to changes in available resources, testing request frequency,and product release cycles. Tests must be chosen and scheduled appropriately,tests must be analyzed for robustness, and testing processes must beappropriately monitored. The proposed work addresses these needs, and byfocusing on software dependability, the work promises to benefit bothproducers (i.e., software developers) and consumers (i.e., the public at large)of software.This work will create new techniques for scheduling test cases duringcontinuous testing based on their historical effectiveness, the relationshipsbetween tests and changes, and their potential for execution; these techniqueswill also be able to adapt to release practices and environmental changes, andwill be supported by analysis techniques that are efficient enough for use inCI testing situations. The work will involve studying the effect of problemsrelated to test design, and developing lightweight static and dynamictechniques for analyzing test cases and test suites to determine whether theymay be problematic. The work will lead to the creation of new approaches fortest monitoring, and the definition of new metrics that capture test and testsuite attributes and states for the assessment of testing results in continuoustesting processes. Finally, the work will result in the creation ofinfrastructure to support rigorous empirical assessments of the techniquesdeveloped.