Research on elastic resource management for multi-queue under cloud computing environment
Research on elastic resource management for multi-queue under cloud computing environment
复制标题
云计算环境下多队列弹性资源管理研究
DOI：
10.1088/1742-6596/898/9/092003
复制
发表时间：
2017-10
期刊：
影响因子：
--
通讯作者：
Cheng Yaodong
中科院分区：
文献类型：
--
作者：
Cheng zhenjing;Li Haibo;Huang Qiulan;Cheng Yaodong
虚拟化技术作为一种管理计算资源的新方法，在高能物理领域得到了越来越广泛的应用。基于Openstack，以HTCondor作为作业队列管理系统，构建了一个虚拟计算集群。在传统的静态集群中，固定数量的虚拟机被预先分配给不同实验的作业队列。然而，这种方法不能很好地适应计算资源需求的波动性。针对这一问题，设计了一种云计算环境下的弹性计算资源管理系统。该系统基于HTCondor的作业队列，采用双资源阈值和配额服务对虚拟计算节点进行统一管理。设计了一个两级池，以提高资源池扩展的效率。本文将介绍IHEPCloud中弹性资源管理系统的几个用例。实际运行表明，虚拟计算资源随着计算需求的变化而动态扩展或收缩。与传统的资源管理方式相比，计算资源的CPU利用率得到了显著提高。当系统中存在多个condor调度器和多个作业队列时，系统也具有良好的性能。
As a new approach to manage computing resource, virtualization technology is more and more widely applied in the high-energy physics field. A virtual computing cluster based on Openstack was built at IHEP, using HTCondor as the job queue management system. In a traditional static cluster, a fixed number of virtual machines are pre-allocated to the job queue of different experiments. However this method cannot be well adapted to the volatility of computing resource requirements. To solve this problem, an elastic computing resource management system under cloud computing environment has been designed. This system performs unified management of virtual computing nodes on the basis of job queue in HTCondor based on dual resource thresholds as well as the quota service. A two-stage pool is designed to improve the efficiency of resource pool expansion. This paper will present several use cases of the elastic resource management system in IHEPCloud. The practical run shows virtual computing resource dynamically expanded or shrunk while computing requirements change. Additionally, the CPU utilization ratio of computing resource was significantly increased when compared with traditional resource management. The system also has good performance when there are multiple condor schedulers and multiple job queues.
登录
查看更多内容
DOI：
10.1109/clustr.2003.1253327
发表时间：
2003-12
期刊：
2003 Proceedings IEEE International Conference on Cluster Computing
影响因子：
--
作者：
Federico D. Sacerdoti;M. Katz;Matthew L. Massie;D. Culler
通讯作者：
Federico D. Sacerdoti;M. Katz;Matthew L. Massie;D. Culler
DOI：
10.1088/1742-6596/664/6/062031
发表时间：
2015-12
期刊：
Journal of Physics: Conference Series
影响因子：
--
作者：
J. Balcas;S. Belforte;B. Bockelman;D. Colling;O. Gutsche;D. Hufnagel;F. Khan;K. Larson;J. Letts;M. Mascheroni;D. Mason;A. McCrea;S. Piperov;M. Saiz-Santos;I. Sfiligoi;A. Tanasijczuk;C. Wissing
通讯作者：
J. Balcas;S. Belforte;B. Bockelman;D. Colling;O. Gutsche;D. Hufnagel;F. Khan;K. Larson;J. Letts;M. Mascheroni;D. Mason;A. McCrea;S. Piperov;M. Saiz-Santos;I. Sfiligoi;A. Tanasijczuk;C. Wissing
DOI：
10.1145/945445.945462
发表时间：
2003-10
期刊：
--
影响因子：
--
作者：
Paul Barham;Boris Dragovic;K. Fraser;S. Hand;T. Harris;Alex Ho;R. Neugebauer;I. Pratt;A. Warfie
通讯作者：
Paul Barham;Boris Dragovic;K. Fraser;S. Hand;T. Harris;Alex Ho;R. Neugebauer;I. Pratt;A. Warfie
DOI：
10.1016/j.future.2012.05.012
发表时间：
2014-03-01
期刊：
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF GRID COMPUTING AND ESCIENCE
影响因子：
--
作者：
Corradi, Antonio;Fanelli, Mario;Foschini, Luca
通讯作者：
Foschini, Luca
DOI：
--
发表时间：
2007-02
期刊：
--
影响因子：
--
作者：
Dave Josephsen
通讯作者：
Dave Josephsen