CAREER: Toward Effective, Predictable, and Consistent Software Testing

每天都有数十亿人依赖于软件系统，但软件仍然存在漏洞和脆弱性。软件测试是软件质量保证（QA）的主要方法，每天都有成千上万的开发人员参与QA活动。开发人员在开发测试时，以及在推断测试的有效性和一致性时，严重依赖于直觉、经验和对代码库的熟悉程度。然而，经验差异很大，这对软件质量和开发人员的生产力有负面影响。为了解决这些挑战，该项目开发了实用的端到端解决方案，用于有效和一致的软件测试，并与现有的QA实践很好地集成在一起。它的目标是通过为开发人员提供具有可量化功效的系统软件测试的框架和方法，从本质上提高渗透到现代技术中的软件的质量。这个项目的总体目标是通过使软件测试更有效、可预测和一致来提高软件质量。为了实现这一目标，拟议的研究共同开发了一种新的基于突变的测试方法以及量化其有效性的方法。具体来说，这个项目(1)开发了程序上下文和测试目标实用工具的模型，(2)建立在这些模型之上，并开发了一种基于学习的方法，用于测试目标实用工具的上下文敏感预测和测试目标排序，以及(3)开发了一种概率方法，用于评估一组测试目标对于软件开发过程中遇到的错误的代表性。除了研究工件之外，这个项目将提供公开可用的工件，使培训新一代软件工程师成为可能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Billions of people depend on software systems every day, but software remains buggy and vulnerable. Software testing is the predominant approach for software quality assurance (QA), and hundreds of thousands of developers are involved in QA activities on a daily basis. Developers heavily rely on intuition, experience, and familiarity with a code base when developing tests, and when reasoning about test effectiveness and consistency. However, experience varies widely, which has negative impacts on software quality and developer productivity. Addressing these challenges, this project develops practical end-to-end solutions for effective and consistent software testing that integrate well with existing QA practices. It aims to substantially increase the quality of the software that pervades modern technology, by providing developers with a framework and methodology for systematic software testing with quantifiable efficacy.The overall goal of this project is to advance software quality by making software testing more effective, predictable, and consistent. To achieve this goal, the proposed research co-develops a new mutation-based testing approach along with a methodology to quantify its efficacy. Specifically, this project (1) develops models for program context and test goal utility, (2) builds upon these models and develops a learning-based approach for context-sensitive predictions of test goal utility and ranking test goals, and (3) develops a probabilistic method that assesses how representative a set of test goals is for bugs encountered during software development. In addition to the research artifacts, this project will provide publicly available artifacts that enable training of a new generation of software engineers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.