Machine Learning for Efficient Regression Testing in Continuous Integration Context

点击翻译按钮获取中文摘要

Continuous Integration (CI) has become one of the main enablers of automated and modern software engineering and has been widely adopted by IT organizations and open-source communities. CI systems allow software developers to submit their code into a shared repository on a frequent basis. Each submission is then built, analyzed, and tested in an automated process. CI provides early feedback to developers, reduces integration risks, and facilitates the early detection of faults. Despite the benefits, CI systems impose a high operational and computational cost that requires significant investment to operate, which threatens the benefits of CI. For example, it has been reported that running CI systems in Google and Microsoft costs these companies millions of dollars. Automated regression testing is crucial to ensure the quality of software systems in CI. However, it is also one of the main reasons for its high costs. For large software projects, software developers need to spend a significant amount of time developing and maintaining many regression test cases (maintainability issues), the execution of which requires significant computational resources (execution issues).   The proposed research program will use state-of-the-art Machine Learning (ML) and software analysis techniques to develop automated, practical, and accurate solutions to improve regression testing efficiency in the context of CI while considering maintainability and execution issues. Test Case Prioritization (TCP) techniques deal with the costly execution of regression testing by prioritizing the specific test cases so that faults may be detected as early as possible. Many TSP techniques have been proposed in the context of regression testing using ML techniques. However, even the best of them cannot achieve satisfactory results for most software systems, arguably due to imbalanced and noisy training datasets. Thus, we will first explore different techniques to account for imbalanced and noisy datasets in order to improve the accuracy of ML-based TCP. We will then explore static analysis, natural language processing, and ML techniques to automatically detect and repair obsolete test cases that are not updated to account for new changes in the software being tested. As, obsolete test cases require significant maintenance efforts from the development team due to the ever-changing codebase in the context of CI. Due to the wide adoption of CI, we believe the result of this research program will help multiple Canadian and international companies decrease the computational and operational costs of their CI systems. This research program will also enable Highly Qualified Personnel (HQP) to acquire practical knowledge on ML, software testing and analysis, and continuous integration. IT organizations increasingly search for HQP who can understand software engineering and ML in order to improve the software development process, ensuring the desirability of HQP from this program.