Test Prioritization and Localization at Ericsson

点击翻译按钮获取中文摘要

Ericsson develops and supplies equipment, software, and services for global mobile networks. Ericsson in Ottawa develops cellular base stations that are the interface between cell phones to the physical backbone networks. This high reliability environment requires testing on expensive hardware that can simulate cell phones interacting with base stations. Some tests are expensive running for days, while others take seconds. When a test fails, the software integration process stops and an engineer must intervene. Our goal is combine test prioritization with empirical software engineering to mine the historical artifacts including past code changes, bugs, peer reviews, and test runs to create statistical models to (1) prioritize tests and (2) help developers categorize and locate the cause of a failure. Research into test prioritization work has been limited by information about past runs and links to other development artifacts. Ericsson collects massive histories of logs, test runs, and development artifacts, in part to provide for reproducibility of software and auditing. Much of the current analysis is done manually where developers must sift through this information to locate the root cause of a failure. My research team has extensive experience in mining and linking software artifacts. Combined with Ericsson's rich data store, we plan to create statistical models and classifiers to efficiently order tests, and to relate other artifacts to provide additional information about the root cause of failures and the factors that contribute to failures. For example, we may find that a certain set of short running tests find over 70% of all failures and that a change to a module that controls signaling by an inexperienced developer will lead to an 10% increase in the likelihood of a high impact failure. These outcomes would increase the productivity of Ericsson Ottawa's 1000 employees and advance the state-of-the-art in understanding which historical measures help in test prioritization and localization.