Testing Cloud Applications under Cloud-Uncertainty Performance Effects
Testing Cloud Applications under Cloud-Uncertainty Performance Effects
复制标题
DOI：
10.1109/icst.2018.00018
复制
发表时间：
2018-04
期刊：
影响因子：
--
通讯作者：
Wei Wang;Ningjing Tian;Sunzhou Huang;Sen He;Abhijeet Srivastava;M. Soffa;L. Pollock
中科院分区：
文献类型：
--
作者：
Wei Wang;Ningjing Tian;Sunzhou Huang;Sen He;Abhijeet Srivastava;M. Soffa;L. Pollock
将应用程序部署到云的范式转变既带来了机遇，也带来了挑战。尽管云利用弹性来扩展运行时的资源使用量，以帮助满足应用程序的性能要求，但开发人员仍然面临性能不可预测、执行环境缺乏控制以及云服务提供商之间差异的挑战，同时还要根据云使用量付费。应用程序性能稳定性尤其受到多租户的影响，其中硬件在不同的应用程序和虚拟机之间共享。开发人员移植应用程序需要满足性能要求，但由于云使用成本高昂，在性能不确定性的影响下在云上进行测试是困难且昂贵的。本文提出了第一种方法，使用典型输入来测试应用程序的性能如何受到性能不确定性的影响，而不会在云中进行强力测试而产生不必要的成本。我们指定云不确定性测试标准，设计基于测试的策略来使用这些测试标准来表征黑盒云的性能分布，并支持执行测试来表征要部署的应用程序的资源使用情况和云基线性能。重要的是，我们开发了一个智能测试预言机，它可以使用上述特征测试结果以一定的置信水平估计应用程序的性能，并确定它是否满足其性能要求。我们在 Chameleon 云和 Amazon Web 服务上评估了我们的测试方法；结果表明，该测试策略有望作为一种经济有效的方法来测试将应用程序移植到云时云不确定性的性能影响。
The paradigm shift of deploying applications to the cloud has introduced both opportunities and challenges. Although clouds use elasticity to scale resource usage at runtime to help meet an application's performance requirements, developers are still challenged by unpredictable performance, little control of execution environment, and differences among cloud service providers, all while being charged for their cloud usages. Application performance stability is particularly affected by multi-tenancy in which the hardware is shared among varying applications and virtual machines. Developers porting their applications need to meet performance requirements, but testing on the cloud under the effects of performance uncertainty is difficult and expensive, due to high cloud usage costs. This paper presents a first approach to testing an application with typical inputs for how its performance will be affected by performance uncertainty, without incurring undue costs of brute force testing in the cloud. We specify cloud uncertainty testing criteria, design a test-based strategy to characterize the black box cloud's performance distributions using these testing criteria, and support execution of tests to characterize the resource usage and cloud baseline performance of the application to be deployed. Importantly, we developed a smart test oracle that estimates the application's performance with certain confidence levels using the above characterization test results and determines whether it will meet its performance requirements. We evaluated our testing approach on both the Chameleon cloud and Amazon web services; results indicate that this testing strategy shows promise as a cost-effective approach to test for performance effects of cloud uncertainty when porting an application to the cloud.