SHF: Medium: Collaborative Research: Regression Testing Techniques for Real-world Software Systems

SHF：媒介：协作研究：现实世界软件系统的回归测试技术

• 批准号: 1161767
• 负责人: Myra Cohen
• 金额: $ 32.49万
• 依托单位: University of Nebraska-Lincoln
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-07-01 至 2017-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1161767&HistoricalAwards=false
• 关键词: SHF; Medium; Collaborative; Research; Regression

Reports estimate that regression testing, which is the activity of retesting a software system after it has been modified, can consume up to 50% of the cost of software development and maintenance. Although there are many techniques that can reduce the cost of regression testing, most of them do not account for important characteristics of modern systems, such as product lines, web applications, service-oriented architectures, and cloud-based applications. These systems are increasingly heterogeneous: they may come from different sources, may be written in different languages, and may be accessible in different formats (e.g., source code, binary code, or through remote interfaces). Moreover, modern software is often environment dependent: its behavior can be affected not only by changes in the code, but also by changes in its complex environment (e.g., databases, configuration files, and network layouts). Because most existing regression-testing techniques do not account for these characteristics, the application of these techniques can result in inadequately tested software, problems during maintenance, and ultimately poor software quality.The overall goal of this research is to go beyond the state of the art in regression testing by defining novel approaches that can be applied to modern, real-world software and account for its characteristics and complexity. To achieve this goal, the research will first extend analysis techniques on which regression-testing approaches rely, such as system modeling, version differencing, coverage analysis, and impact analysis. The research will then leverage these fundamental techniques to develop, evaluate with industrial partners, and make available a family of regression testing techniques and tools that can (1) build comprehensive models of heterogeneous, environment-dependent software systems, (2) evolve these models throughout the systems' lifetimes, and (3) analyze the changes across models to understand their effects on the systems' behavior and retest them effectively and efficiently. The impact of the research will be manyfold. First, the rigorous, transformative, and highly automated techniques developed will help improve the quality of today's large, complex software systems, thus benefitting all segments of society that depend on software. Second, the release of the produced tools and infrastructure will let other researchers and practitioners build on our results, advancing knowledge and understanding. Finally, the research findings will be integrated in curriculum materials that will be made available to the broader scientific community, which will help prepare a globally competitive workforce and further benefit society.

报告估计，回归测试（即软件系统修改后重新测试的活动）可能会消耗高达 50% 的软件开发和维护成本。尽管有很多技术可以降低回归测试的成本，但大多数技术都没有考虑到现代系统的重要特征，例如产品线、Web 应用程序、面向服务的架构和基于云的应用程序。 这些系统越来越异构：它们可能来自不同的来源，可能用不同的语言编写，并且可以以不同的格式访问（例如，源代码、二进制代码或通过远程接口）。此外，现代软件通常依赖于环境：其行为不仅会受到代码变化的影响，还会受到复杂环境（例如数据库、配置文件和网络布局）变化的影响。 由于大多数现有的回归测试技术没有考虑到这些特征，因此应用这些技术可能会导致软件测试不充分、维护过程中出现问题，并最终导致软件质量下降。这项研究的总体目标是通过定义可应用于现代现实世界软件并解释其特征和复杂性的新颖方法来超越回归测试的现有技术。为了实现这一目标，该研究将首先扩展回归测试方法所依赖的分析技术，例如系统建模、版本差异、覆盖率分析和影响分析。然后，该研究将利用这些基本技术与工业合作伙伴一起开发、评估，并提供一系列回归测试技术和工具，这些技术和工具可以（1）构建异构的、依赖于环境的软件系统的综合模型，（2）在系统的整个生命周期中发展这些模型，以及（3）分析模型之间的变化，以了解它们对系统行为的影响，并有效和高效地重新测试它们。这项研究的影响将是多方面的。首先，所开发的严格的、变革性的、高度自动化的技术将有助于提高当今大型、复杂的软件系统的质量，从而使依赖软件的社会各阶层受益。其次，所生产的工具和基础设施的发布将使其他研究人员和从业者能够以我们的成果为基础，推进知识和理解。最后，研究结果将纳入课程材料中，向更广泛的科学界提供，这将有助于培养具有全球竞争力的劳动力并进一步造福社会。