Analyzing Tests for Correctness, Adequacy, and Effectiveness

点击翻译按钮获取中文摘要

Software developers continuously apply code changes to add new features, improve the product, or fix known bugs. For instance, 50% of the Google code base is reported to change every month. In order to ensure that the code base remains healthy, test cases are written to verify that the production code functions as expected. Such tests also serve as continuous regression tests to ensure that previously working functionality still works, when the software evolves. ***Inadequate testing costs the US economy $60 billion annually. A more recent study by Cambridge University finds that software bugs cost the global economy $312 billion per year. The study also found that, on average, software developers spend 50% of their programming time finding and fixing bugs. ***Test cases are meant to form the first line of defence against the introduction of faults in a software product, especially when retesting modified code. Therefore, assessing the quality of a test suite is crucial to ensure that software systems are correctly implemented, which can help to decrease the overall development and maintenance cost.***Test quality can be considered in different perspectives. We propose to study the correctness, adequacy, and effectiveness of test cases, which directly impact the overall test suite quality. The objectives of this research proposal are to (1) study test quality issues related to test correctness, test adequacy, and test effectiveness in practice, (2) explore novel techniques for analyzing and assessing the quality of test cases, (3) and propose new solutions for increasing the quality of test cases directly, and the production code being tested as a result. ***Although the quality of test code is as important as production code, unlike production code, the quality of tests written by developers has not received much attention from the research community thus far, especially when it comes to the correctness of test cases. This proposal seeks to study and analyze the quality of tests in depth. The results of this work will have positive implications for both software testers and developers. The techniques proposed will make test cases more correct, adequate, and effective and as a result the quality of the production code can be improved and the software can be released with much more confident to end-users.**