CAREER: Toward Effective, Predictable, and Consistent Software Testing

每天有数十亿人依赖于软件系统，但软件仍然存在缺陷和脆弱性。软件测试是软件质量保证（QA）的主要方法，每天都有成千上万的开发人员参与QA活动。开发人员在开发测试时，以及在推理测试的有效性和一致性时，严重依赖于直觉、经验和对代码库的熟悉。然而，经验差异很大，这对软件质量和开发人员的生产力产生了负面影响。为了应对这些挑战，该项目开发了实用的端到端解决方案，用于有效和一致的软件测试，并与现有的QA实践很好地集成。它旨在通过为开发人员提供一个具有可量化功效的系统软件测试框架和方法，大幅提高现代技术中普遍存在的软件质量。该项目的总体目标是通过使软件测试更有效，可预测和一致来提高软件质量。为了实现这一目标，拟议的研究共同开发了一种新的基于突变的测试方法，沿着一种方法来量化其功效。具体来说，这个项目（1）开发模型的程序上下文和测试目标的效用，（2）建立在这些模型和开发一个基于学习的方法，上下文敏感的预测测试目标的效用和排名测试目标，（3）开发一个概率方法，评估如何代表一组测试目标是在软件开发过程中遇到的错误。除了研究成果外，该项目还将提供可公开获得的成果，以培训新一代的软件工程师。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Billions of people depend on software systems every day, but software remains buggy and vulnerable. Software testing is the predominant approach for software quality assurance (QA), and hundreds of thousands of developers are involved in QA activities on a daily basis. Developers heavily rely on intuition, experience, and familiarity with a code base when developing tests, and when reasoning about test effectiveness and consistency. However, experience varies widely, which has negative impacts on software quality and developer productivity. Addressing these challenges, this project develops practical end-to-end solutions for effective and consistent software testing that integrate well with existing QA practices. It aims to substantially increase the quality of the software that pervades modern technology, by providing developers with a framework and methodology for systematic software testing with quantifiable efficacy.The overall goal of this project is to advance software quality by making software testing more effective, predictable, and consistent. To achieve this goal, the proposed research co-develops a new mutation-based testing approach along with a methodology to quantify its efficacy. Specifically, this project (1) develops models for program context and test goal utility, (2) builds upon these models and develops a learning-based approach for context-sensitive predictions of test goal utility and ranking test goals, and (3) develops a probabilistic method that assesses how representative a set of test goals is for bugs encountered during software development. In addition to the research artifacts, this project will provide publicly available artifacts that enable training of a new generation of software engineers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.