An Empirical Analysis of UI-Based Flaky Tests
An Empirical Analysis of UI-Based Flaky Tests
复制标题
DOI：
10.1109/icse43902.2021.00141
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Alan Romano;Zihe Song;Sampath Grandhi;Wei Yang;Weihang Wang
中科院分区：
文献类型：
--
作者：
Alan Romano;Zihe Song;Sampath Grandhi;Wei Yang;Weihang Wang
近年来，薄片测试受到了研究界的关注，这是有充分理由的。这些测试会浪费时间和资源，降低测试套件的可靠性并构建受其影响的系统。然而，现有的大部分关于薄片测试的工作都只关注传统的单元测试。这项工作忽略了与传统单元测试相比具有更大输入空间和更多样化运行条件的UI测试。此外，UI测试往往更复杂、更耗费资源，这使得它们不适合涉及多次重新运行测试套件的检测技术。在本文中，我们对片状用户界面测试进行了研究。我们分析了来自Web和Android环境的62个项目中发现的235个不可靠的UI测试样本。我们确定了UI测试中常见的剥落根本原因，用于显示剥落行为的策略，以及用于修复剥落的UI测试的修复策略。这项工作的发现可以为开发用户界面测试中出现的片状现象的检测和预防技术提供基础。
Flaky tests have gained attention from the research community in recent years and with good reason. These tests lead to wasted time and resources, and they reduce the reliability of the test suites and build systems they affect. However, most of the existing work on flaky tests focus exclusively on traditional unit tests. This work ignores UI tests that have larger input spaces and more diverse running conditions than traditional unit tests. In addition, UI tests tend to be more complex and resource-heavy, making them unsuited for detection techniques involving rerunning test suites multiple times. In this paper, we perform a study on flaky UI tests. We analyze 235 flaky UI test samples found in 62 projects from both web and Android environments. We identify the common underlying root causes of flakiness in the UI tests, the strategies used to manifest the flaky behavior, and the fixing strategies used to remedy flaky UI tests. The findings made in this work can provide a foundation for the development of detection and prevention techniques for flakiness arising in UI tests.