Testing Configuration Changes in Context to Prevent Production Failures
Testing Configuration Changes in Context to Prevent Production Failures
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Xudong Sun;Runxiang Cheng;Jianyan Chen;Elaine Ang;Owolabi Legunsen;Tianyin Xu
中科院分区：
文献类型：
--
作者：
Xudong Sun;Runxiang Cheng;Jianyan Chen;Elaine Ang;Owolabi Legunsen;Tianyin Xu
大规模的云服务每天向生产系统部署数百个配置更改。在这样的速度下，配置变化不可避免地成为生产失败的主要原因。现有的错误配置检测和配置验证技术仅检查配置值。这些技术无法检测常见类型的导致失败的配置更改，例如导致代码失败的更改或违反隐藏约束的更改。我们提出了一种用于检测故障的新型测试--CATS，用于检测导致配置更改的故障，以防止生产故障。Ctest背后的思想很简单--将生产系统配置连接到软件测试，以便可以在受更改影响的代码环境中测试配置更改。因此，ctest可以检测到暴露休眠软件错误和各种错误配置的配置更改。我们展示了如何通过转换成熟系统中的许多现有测试来生成测试。我们要解决的关键挑战是自动识别可在测试中重用的测试逻辑和先知。我们从五个云系统中的现有测试生成了数千个测试。我们的结果表明，ctest在部署前有效地检测到导致失败的配置更改。我们对真实世界中导致故障的配置更改、注入的错误配置以及从公共Docker映像部署的配置文件进行测试。测试有效地检测真实世界中导致失败的配置更改、各种注入的错误配置以及部署文件中的错误配置。
Large-scale cloud services deploy hundreds of configuration changes to production systems daily. At such velocity, configuration changes have inevitably become prevalent causes of production failures. Existing misconfiguration detection and configuration validation techniques only check configuration values. These techniques cannot detect common types of failure-inducing configuration changes, such as those that cause code to fail or those that violate hidden constraints. We present ctests, a new type of tests for detecting failureinducing configuration changes to prevent production failures. The idea behind ctests is simple—connecting production system configurations to software tests so that configuration changes can be tested in the context of code affected by the changes. So, ctests can detect configuration changes that expose dormant software bugs and diverse misconfigurations. We show how to generate ctests by transforming the many existing tests in mature systems. The key challenge that we address is the automated identification of test logic and oracles that can be reused in ctests. We generated thousands of ctests from the existing tests in five cloud systems. Our results show that ctests are effective in detecting failure-inducing configuration changes before deployment. We evaluate ctests on real-world failure-inducing configuration changes, injected misconfigurations, and deployed configuration files from public Docker images. Ctests effectively detect real-world failure-inducing configuration changes, diverse injected misconfigurations and misconfigurations in the deployed files.