SHF: Medium: Collaborative Research: Regression Testing Techniques for Real-world Software Systems

报告估计，回归测试是在修改软件系统后重新测试软件系统的活动，可以消耗高达50%的软件开发和维护成本。虽然有许多技术可以降低回归测试的成本，但它们中的大多数都没有考虑到现代系统的重要特征，如产品线、Web应用程序、面向服务的体系结构和基于云的应用程序。这些系统的异构性越来越强：它们可能来自不同的来源，可能用不同的语言编写，并且可能以不同的格式访问(例如，源代码、二进制代码或通过远程接口)。此外，现代软件通常依赖于环境：其行为不仅会受到代码更改的影响，还会受到其复杂环境(例如数据库、配置文件和网络布局)的更改的影响。由于大多数现有的回归测试技术没有考虑到这些特征，这些技术的应用可能导致软件测试不足，维护过程中出现问题，最终导致软件质量低下。本研究的总体目标是通过定义可应用于现代现实世界软件的新方法，并考虑其特征和复杂性，来超越回归测试的最新水平。为了实现这一目标，研究将首先扩展回归测试方法所依赖的分析技术，如系统建模、版本差异、覆盖分析和影响分析。然后，研究将利用这些基本技术与行业合作伙伴一起开发和评估，并提供一系列回归测试技术和工具，这些技术和工具可以(1)构建与环境相关的异类软件系统的综合模型，(2)在系统的整个生命周期中发展这些模型，以及(3)分析跨模型的变化，以了解它们对系统行为的影响，并有效和高效地重新测试它们。这项研究的影响是多方面的。首先，开发的严谨、变革性和高度自动化的技术将有助于提高当今大型、复杂软件系统的质量，从而使依赖软件的所有社会阶层受益。其次，生产的工具和基础设施的发布将让其他研究人员和从业者在我们成果的基础上再接再厉，增进知识和理解。最后，研究结果将被整合到课程材料中，供更广泛的科学界使用，这将有助于培养一支具有全球竞争力的劳动力队伍，并进一步造福社会。

Reports estimate that regression testing, which is the activity of retesting a software system after it has been modified, can consume up to 50% of the cost of software development and maintenance. Although there are many techniques that can reduce the cost of regression testing, most of them do not account for important characteristics of modern systems, such as product lines, web applications, service-oriented architectures, and cloud-based applications. These systems are increasingly heterogeneous: they may come from different sources, may be written in different languages, and may be accessible in different formats (e.g., source code, binary code, or through remote interfaces). Moreover, modern software is often environment dependent: its behavior can be affected not only by changes in the code, but also by changes in its complex environment (e.g., databases, configuration files, and network layouts). Because most existing regression-testing techniques do not account for these characteristics, the application of these techniques can result in inadequately tested software, problems during maintenance, and ultimately poor software quality.The overall goal of this research is to go beyond the state of the art in regression testing by defining novel approaches that can be applied to modern, real-world software and account for its characteristics and complexity. To achieve this goal, the research will first extend analysis techniques on which regression-testing approaches rely, such as system modeling, version differencing, coverage analysis, and impact analysis. The research will then leverage these fundamental techniques to develop, evaluate with industrial partners, and make available a family of regression testing techniques and tools that can (1) build comprehensive models of heterogeneous, environment-dependent software systems, (2) evolve these models throughout the systems' lifetimes, and (3) analyze the changes across models to understand their effects on the systems' behavior and retest them effectively and efficiently. The impact of the research will be manyfold. First, the rigorous, transformative, and highly automated techniques developed will help improve the quality of today's large, complex software systems, thus benefitting all segments of society that depend on software. Second, the release of the produced tools and infrastructure will let other researchers and practitioners build on our results, advancing knowledge and understanding. Finally, the research findings will be integrated in curriculum materials that will be made available to the broader scientific community, which will help prepare a globally competitive workforce and further benefit society.