Analyzing causes of test failures

在这项工作中，我们建议进行系统的分析测试失败，并提出策略，以减轻测试片状。这项工作将调查在测试执行日志中观察到的症状，以及潜在的原因，包括最近的代码更改;测试依赖项的更改;以及插件加载顺序。它还将调查测试类型和失败率之间的相关性：例如，Arista的一些测试是基于状态机的。这样的测试失败的次数比其他测试多还是少？ 这项工作的直接结果将是对失败测试的当前状态的诊断，沿着增加测试套件可靠性的建议。我们还建议建立模型的测试用例，这将封装的原因，他们失败。

In this work, we propose to carry out a systematic analysis of test failures and suggest strategies to mitigate test flakiness. The work would investigate symptoms as observed in test execution logs, as well as potential causes including recent code changes; changes in tests' dependencies; and plugin load orders. It would also investigate correlations between test types and failure rates: for instance, some of Arista's tests are based on state machines. Do such tests fail more or less often than other tests? The immediate outcome of the work would be a diagnosis of the current state of failing tests along with suggestions to increase the reliability of the test suite. We also propose to build models of test cases which would encapsulate the reasons that they fail.