SHF: Small:Testing in the Presence of Continuous Change

许多开发软件的组织正在利用持续集成过程，在这个过程中，工程师经常将代码与主线代码库合并，并不断地对其进行回归测试。在这样的环境中，进化的速度和必须运行的测试数量可能是惊人的。为了确保这些情况下的软件质量，测试过程需要快速适应可用资源、测试请求频率和产品发布周期的变化。必须适当地选择和安排测试，必须对测试进行鲁棒性分析，并且必须对测试过程进行适当的监控。建议的工作解决了这些需求，并且通过关注软件的可靠性，该工作承诺使软件的生产者（即软件开发人员）和消费者（即广大公众）都受益。这项工作将创建新的技术，用于在连续测试期间根据它们的历史有效性、测试和变更之间的关系以及它们执行的潜力来调度测试用例；这些技术也将能够适应发布实践和环境变化，并且将得到分析技术的支持，这些分析技术对于使用inCI测试情况是足够有效的。这项工作将包括研究与测试设计相关的问题的影响，以及开发用于分析测试用例和测试套件的轻量级静态和动态技术，以确定它们是否可能存在问题。这项工作将导致创建新的测试监控方法，并定义新的度量，这些度量捕获测试和测试套件属性，以及持续测试过程中评估测试结果的状态。最后，这项工作将导致基础设施的建立，以支持对所开发的技术进行严格的实证评估。

Many organizations that develop software are utilizing continuous integrationprocesses, in which engineers merge code frequently with the mainline codebase,and continuously regression test it. The pace at which evolution occurs insuch environments, and the numbers of tests that must be run, can be staggering.To ensure the quality of software in these situations, testing processes needto adapt quickly to changes in available resources, testing request frequency,and product release cycles. Tests must be chosen and scheduled appropriately,tests must be analyzed for robustness, and testing processes must beappropriately monitored. The proposed work addresses these needs, and byfocusing on software dependability, the work promises to benefit bothproducers (i.e., software developers) and consumers (i.e., the public at large)of software.This work will create new techniques for scheduling test cases duringcontinuous testing based on their historical effectiveness, the relationshipsbetween tests and changes, and their potential for execution; these techniqueswill also be able to adapt to release practices and environmental changes, andwill be supported by analysis techniques that are efficient enough for use inCI testing situations. The work will involve studying the effect of problemsrelated to test design, and developing lightweight static and dynamictechniques for analyzing test cases and test suites to determine whether theymay be problematic. The work will lead to the creation of new approaches fortest monitoring, and the definition of new metrics that capture test and testsuite attributes and states for the assessment of testing results in continuoustesting processes. Finally, the work will result in the creation ofinfrastructure to support rigorous empirical assessments of the techniquesdeveloped.