Building Semi-Elastic Virtual Clusters for Cost-Effective HPC Cloud Resource Provisioning
Building Semi-Elastic Virtual Clusters for Cost-Effective HPC Cloud Resource Provisioning
复制标题
构建半弹性虚拟集群以实现经济高效的HPC云资源配置
DOI：
10.1109/tpds.2015.2476459
复制
发表时间：
2016
影响因子：
5.3
通讯作者：
Zheng Weimin
中科院分区：
文献类型：
--
作者：
Niu Shuangcheng;Zhai Jidong;Ma Xiaosong;Tang Xiongchao;Chen Wenguang;Zheng Weimin
最近的研究发现，云环境越来越适合执行HPC应用程序，包括紧密耦合的并行模拟。与此同时，尽管公共云提供弹性的按需资源调配和按需付费定价，但设置按需虚拟集群的个人用户可能无法充分利用常见的节省成本的机会，例如预留实例。本文提出了一种半弹性集群计算模型，用于组织预留和动态调整基于云的虚拟集群的规模。我们提出了一套由SEC独有的集成批处理调度和资源伸缩策略，以及基于作业历史的在线预留实例供应算法。我们的跟踪驱动模拟结果表明，这种模型比单个用户获取和管理云资源节省了61.0%的成本，而不会导致平均作业等待时间延长。此外，为了利用不同公有云的优势，我们还将SEC扩展到多云环境，在这种环境中，SEC可以获得比任何单一云上更低的成本。我们设计并实现了一个SEC模型的原型系统，并从管理开销和平均作业等待时间两个方面对其进行了评估。实验结果表明，与作业等待时间相比，管理开销可以忽略不计。
Recent studies have found cloud environments increasingly appealing for executing HPC applications, including tightly coupled parallel simulations. At the same time, while public clouds offer elastic, on-demand resource provisioning and pay-as-you-go pricing, individual users setting up their on-demand virtual clusters may not be able to take full advantage of common cost-saving opportunities, such as reserved instances. In this paper, we propose a Semi-Elastic Cluster (SEC) computing model for organizations to reserve and dynamically resize a virtual cloud-based cluster. We present a set of integrated batch scheduling plus resource scaling strategies uniquely enabled by SEC, as well as an online reserved instance provisioning algorithm based on job history. Our trace-driven simulation results show that such a model has a 61.0 percent cost saving than individual users acquiring and managing cloud resources without causing longer average job wait time. Moreover, to exploit the advantages of different public clouds, we also extend SEC to a multi-cloud environment, where SEC can get a lower cost than on any single cloud. We design and implement a prototype system of the SEC model and evaluate it in terms of management overhead and average job wait time. Experimental results show that the management overhead is negligible with respect to the job wait time.