Code Coverage Criteria for Asynchronous Programs
Code Coverage Criteria for Asynchronous Programs
复制标题
DOI：
10.1145/3611643.3616292
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Mohammad Ganji;Saba Alimadadi;Frank Tip
中科院分区：
文献类型：
--
作者：
Mohammad Ganji;Saba Alimadadi;Frank Tip
异步软件通常表现出复杂和易出错的行为，应该进行彻底的测试。代码覆盖率是评估测试套件质量的最流行的度量。然而，传统的代码覆盖标准不能充分反映异步操作的完成、交互和错误处理。本文提出了新的测试充分性标准来衡量：(i)在成功和异常执行方面完成异步操作，（ii）注册处理两种可能结果的反应，以及（iii）通过测试执行上述反应。我们实现了JScope，一个根据JavaScript应用程序中的这些标准自动测量覆盖率的工具，作为Visual Studio Code的交互式插件。对20个JavaScript应用程序的JScope的评估表明，建议的标准可以帮助改进测试充分性的评估，补充传统的标准。根据我们对15个与异步相关的真实GitHub问题的调查，新的标准可以帮助揭示那些未经测试但被传统覆盖标准所涵盖的错误异步行为。我们还报告了一个有12名参与者的对照实验，以调查JScope在实际设置中的有用性，证明它在提高程序员评估测试充分性和检测异步代码的未测试行为的能力方面的有效性。
Asynchronous software often exhibits complex and error-prone behaviors that should be tested thoroughly. Code coverage has been the most popular metric to assess test suite quality. However, traditional code coverage criteria do not adequately reflect completion, interactions, and error handling of asynchronous operations. This paper proposes novel test adequacy criteria for measuring: (i) completion of asynchronous operations in terms of both successful and exceptional execution, (ii) registration of reactions for handling both possible outcomes, and (iii) execution of said reactions through tests. We implement JScope, a tool for automatically measuring coverage according to these criteria in JavaScript applications, as an interactive plug-in for Visual Studio Code. An evaluation of JScope on 20 JavaScript applications shows that the proposed criteria can help improve assessment of test adequacy, complementing traditional criteria. According to our investigation of 15 real GitHub issues concerned with asynchrony, the new criteria can help reveal faulty asynchronous behaviors that are untested yet are deemed covered by traditional coverage criteria. We also report on a controlled experiment with 12 participants to investigate the usefulness of JScope in realistic settings, demonstrating its effectiveness in improving programmers’ ability to assess test adequacy and detect untested behavior of asynchronous code.