Analyzing Tests for Correctness, Adequacy, and Effectiveness

分析测试的正确性、充分性和有效性

• 批准号: RGPIN-2016-04615
• 负责人: Mesbah, Ali
• 金额: $ 3.35万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=689216
• 关键词: Analyzing; Tests; Correctness; Adequacy; Effectiveness

Software developers continuously apply code changes to add new features, improve the product, or fix known bugs. For instance, 50% of the Google code base is reported to change every month. In order to ensure that the code base remains healthy, test cases are written to verify that the production code functions as expected. Such tests also serve as continuous regression tests to ensure that previously working functionality still works, when the software evolves. ***Inadequate testing costs the US economy $60 billion annually. A more recent study by Cambridge University finds that software bugs cost the global economy $312 billion per year. The study also found that, on average, software developers spend 50% of their programming time finding and fixing bugs. ***Test cases are meant to form the first line of defence against the introduction of faults in a software product, especially when retesting modified code. Therefore, assessing the quality of a test suite is crucial to ensure that software systems are correctly implemented, which can help to decrease the overall development and maintenance cost.***Test quality can be considered in different perspectives. We propose to study the correctness, adequacy, and effectiveness of test cases, which directly impact the overall test suite quality. The objectives of this research proposal are to (1) study test quality issues related to test correctness, test adequacy, and test effectiveness in practice, (2) explore novel techniques for analyzing and assessing the quality of test cases, (3) and propose new solutions for increasing the quality of test cases directly, and the production code being tested as a result. ***Although the quality of test code is as important as production code, unlike production code, the quality of tests written by developers has not received much attention from the research community thus far, especially when it comes to the correctness of test cases. This proposal seeks to study and analyze the quality of tests in depth. The results of this work will have positive implications for both software testers and developers. The techniques proposed will make test cases more correct, adequate, and effective and as a result the quality of the production code can be improved and the software can be released with much more confident to end-users.**

软件开发人员不断地应用代码更改来添加新功能、改进产品或修复已知错误。例如，据报道，50%的谷歌代码库每月都会发生变化。为了确保代码库保持健康，编写了测试用例以验证生产代码按预期运行。这样的测试还充当连续回归测试，以确保当软件发展时，以前工作的功能仍然有效。*不充分的测试每年给美国经济造成600亿美元的损失。剑桥大学最近的一项研究发现，软件漏洞每年给全球经济造成3120亿美元的损失。研究还发现，平均而言，软件开发人员将50%的编程时间用于查找和修复错误。*测试用例旨在形成第一道防线，防止在软件产品中引入错误，特别是在重新测试修改后的代码时。因此，评估测试套件的质量对于确保软件系统的正确实现至关重要，这有助于降低整体开发和维护成本。*测试质量可以从不同的角度进行考虑。我们建议研究测试用例的正确性、充分性和有效性，这将直接影响整个测试用例的质量。该研究方案的目的是(1)研究与测试正确性、测试充分性和测试有效性相关的测试质量问题；(2)探索分析和评估测试用例质量的新技术；(3)提出新的解决方案，直接提高测试用例的质量，并由此测试生成的代码。*虽然测试代码的质量和生产代码一样重要，但与生产代码不同，开发人员编写的测试的质量到目前为止还没有受到研究界的太多关注，特别是在测试用例的正确性方面。这项建议旨在深入研究和分析测试的质量。这项工作的结果将对软件测试人员和开发人员都有积极的影响。所提出的技术将使测试用例更加正确、充分和有效，从而可以提高生产代码的质量，并且可以更有信心地向最终用户发布软件。**