Machine Learning for Efficient Regression Testing in Continuous Integration Context

持续集成（CI）已经成为自动化和现代软件工程的主要推动者之一，并已被IT组织和开源社区广泛采用。CI系统允许软件开发人员经常将他们的代码提交到共享存储库中。然后在自动化过程中构建、分析和测试每个提交。CI为开发人员提供早期反馈，降低集成风险，并促进早期发现故障。尽管有这些好处，但是CI系统施加了很高的操作和计算成本，需要大量的投资来操作，这威胁到CI的好处。例如，据报道，在b谷歌和微软中运行CI系统花费了这些公司数百万美元。自动化回归测试对于确保CI中软件系统的质量至关重要。然而，这也是其高成本的主要原因之一。对于大型软件项目，软件开发人员需要花费大量的时间来开发和维护许多回归测试用例（可维护性问题），其执行需要大量的计算资源（执行问题）。拟议的研究计划将使用最先进的机器学习（ML）和软件分析技术来开发自动化，实用和准确的解决方案，以提高CI背景下的回归测试效率，同时考虑可维护性和执行问题。测试用例优先级（TCP）技术通过对特定的测试用例进行优先级排序，以便尽可能早地检测到错误，从而处理回归测试的昂贵执行。许多TSP技术已经在使用ML技术的回归测试的背景下被提出。然而，即使是最好的方法也不能在大多数软件系统中获得令人满意的结果，这可能是由于训练数据集的不平衡和噪声。因此，我们将首先探索不同的技术来解释不平衡和有噪声的数据集，以提高基于ml的TCP的准确性。然后，我们将探索静态分析、自然语言处理和ML技术，以自动检测和修复过时的测试用例，这些测试用例没有更新，以说明正在测试的软件中的新变化。由于CI环境中不断变化的代码库，过时的测试用例需要开发团队进行大量的维护工作。由于CI的广泛采用，我们相信该研究项目的结果将帮助多家加拿大和国际公司降低CI系统的计算和运营成本。该研究项目还将使高素质人才（HQP）获得机器学习、软件测试和分析以及持续集成方面的实用知识。IT组织越来越多地寻找能够理解软件工程和机器学习的HQP，以改进软件开发过程，确保HQP从这个项目中脱颖而出。

Continuous Integration (CI) has become one of the main enablers of automated and modern software engineering and has been widely adopted by IT organizations and open-source communities. CI systems allow software developers to submit their code into a shared repository on a frequent basis. Each submission is then built, analyzed, and tested in an automated process. CI provides early feedback to developers, reduces integration risks, and facilitates the early detection of faults. Despite the benefits, CI systems impose a high operational and computational cost that requires significant investment to operate, which threatens the benefits of CI. For example, it has been reported that running CI systems in Google and Microsoft costs these companies millions of dollars. Automated regression testing is crucial to ensure the quality of software systems in CI. However, it is also one of the main reasons for its high costs. For large software projects, software developers need to spend a significant amount of time developing and maintaining many regression test cases (maintainability issues), the execution of which requires significant computational resources (execution issues).   The proposed research program will use state-of-the-art Machine Learning (ML) and software analysis techniques to develop automated, practical, and accurate solutions to improve regression testing efficiency in the context of CI while considering maintainability and execution issues. Test Case Prioritization (TCP) techniques deal with the costly execution of regression testing by prioritizing the specific test cases so that faults may be detected as early as possible. Many TSP techniques have been proposed in the context of regression testing using ML techniques. However, even the best of them cannot achieve satisfactory results for most software systems, arguably due to imbalanced and noisy training datasets. Thus, we will first explore different techniques to account for imbalanced and noisy datasets in order to improve the accuracy of ML-based TCP. We will then explore static analysis, natural language processing, and ML techniques to automatically detect and repair obsolete test cases that are not updated to account for new changes in the software being tested. As, obsolete test cases require significant maintenance efforts from the development team due to the ever-changing codebase in the context of CI. Due to the wide adoption of CI, we believe the result of this research program will help multiple Canadian and international companies decrease the computational and operational costs of their CI systems. This research program will also enable Highly Qualified Personnel (HQP) to acquire practical knowledge on ML, software testing and analysis, and continuous integration. IT organizations increasingly search for HQP who can understand software engineering and ML in order to improve the software development process, ensuring the desirability of HQP from this program.