Test Effectiveness, Localization, Prioritization, and Risk in Ericssons's Complex Test Environment

爱立信复杂测试环境中的测试有效性、本地化、优先级和风险

• 批准号: 502012-2016
• 负责人: Rigby, Peter
• 金额: $ 4.32万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=663372
• 关键词: Test; Effectiveness; Localization; Prioritization; Risk

Ericsson develops and supplies equipment, software, and services for global mobile networks. Ericsson in Ottawa develops cellular base stations that are the interface between cell phones and the physical backbone networks. This high reliability environment requires testing on expensive hardware that can simulate cell phones interacting with base stations. Some tests are expensive running for days, while others take seconds. When a test fails, the software integration process stops and an engineer must intervene. Our goal is combine test prioritization, test log analysis, and empirical software engineering to mine the historical artifacts including past code changes, bugs, peer reviews, and test runs to create statistical models (1) to determine effective environmentally valid test prioritizations, (2) to differentiate between environment and product test failures, (3) to locate faults by cleaning the noise out of test logs, (4) to quantify change risk and identify software units that need new tests, and (5) to integrate these actionable results into our proposed tool called TestGuru. **Research into test prioritization work has been limited by information about past runs and links to other development artifacts. Ericsson collects massive histories of logs and test runs in part to provide for reproducibility of software and auditing. The research team has extensive experience in mining and linking software artifacts including logs and tests. Combined with Ericsson's rich data store, we plan to create statistical models and classifiers to efficiently order tests and quantify risk, and to link artifacts to provide additional information about the root cause of failures and the factors that contribute to failures. Through an Engage and MITACS, we provide testers with ranked lists of anomalous tests that they use each morning to prioritize the tests they will investigate. By mining the logs we will be able to help developers isolate the currently failing code. By mining version history we will be able to identify risky changes that may need more testing. These outcomes will increase the productivity of Ericsson Ottawa's 1000 employees and advance the state-of-the-art in testing.****

爱立信为全球移动的网络开发和提供设备、软件和服务。位于渥太华的爱立信开发了蜂窝基站，它是手机和物理骨干网络之间的接口。这种高可靠性环境需要在昂贵的硬件上进行测试，这些硬件可以模拟手机与基站的交互。有些测试运行几天是昂贵的，而另一些只需要几秒钟。当测试失败时，软件集成过程停止，工程师必须介入。我们的目标是结合联合收割机测试优先级，测试日志分析和经验软件工程来挖掘历史工件，包括过去的代码更改，错误，同行评审和测试运行，以创建统计模型（1）确定有效的环境有效的测试优先级，（2）区分环境和产品测试失败，（3）通过清除测试日志中的噪音来定位故障，（4）量化变更风险并识别需要新测试的软件单元，以及（5）将这些可操作的结果集成到我们提议的名为TestGuru的工具中。** 对测试优先级工作的研究受到过去运行信息和其他开发工件链接的限制。爱立信收集大量的日志和测试运行历史，部分是为了提供软件和审计的可重复性。该研究团队在挖掘和链接软件工件（包括日志和测试）方面拥有丰富的经验。结合爱立信丰富的数据存储，我们计划创建统计模型和分类器，以有效地排序测试和量化风险，并链接工件，以提供有关故障根本原因和导致故障的因素的额外信息。通过Engage和MITACS，我们为测试人员提供了异常测试的排名列表，他们每天早上使用这些列表来确定他们将调查的测试的优先级。通过挖掘日志，我们将能够帮助开发人员隔离当前失败的代码。通过挖掘版本历史，我们将能够识别可能需要更多测试的风险更改。这些成果将提高爱立信渥太华1000名员工的生产力，并推动测试的最新发展。*