Cloud Bursting with GlideinWMS: Means to satisfy ever increasing computing needs for Scientific Workflows
Cloud Bursting with GlideinWMS: Means to satisfy ever increasing computing needs for Scientific Workflows
复制标题
使用 GlideinWMS 进行云爆发：满足科学工作流程不断增长的计算需求的方法
DOI：
--
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
M. Rynge
中科院分区：
文献类型：
--
作者：
P. Mhashilkar;A. Tiradani;B. Holzman;K. Larson;I. Sfiligoi;M. Rynge
科学界一直处于采用新技术和计算方法的最前沿。科学计算已经影响了今天的科学，实现了几十年前不可能实现的突破。在过去的十年中，开放科学网格（OSG）和欧洲网格基础设施（EGI）中的几个这样的社区一直在使用GlideinWMS来运行复杂的应用程序工作流，以有效地共享网格上的计算资源。GlideinWMS是一个基于试验的工作负载管理系统（WMS），它可以根据需要在网格资源上创建一个动态大小的覆盖HTCondor批处理系统。目前，网格上共享的计算资源仅能满足计算需求。我们设想，由“大数据”驱动的科学的复杂性将进一步推动对计算资源的需求。为了满足他们不断增长的需求和/或运行专业化的工作流程，一些大型社区（如CMS）正在研究使用云计算作为云结构即服务（IAAS），GlideinWMS作为填补空白的潜在替代方案。类似地，以前没有访问计算资源的社区可以使用GlideinWMS在云基础设施上设置批处理系统。为了实现这一点，GlideinWMS的架构已经扩展到支持GlideinWMS与不同的科学和商业云提供商（如HLT，FutureGrid，FermiCloud和Amazon EC2）的接口。在本文中，我们描述了一个解决方案的云爆发与滑翔WMS。本文描述了在GlideinWMS中支持云基础设施的方法、架构变化和经验教训。
Scientific communities have been in the forefront of adopting new technologies and methodologies in the computing. Scientific computing has influenced how science is done today, achieving breakthroughs that were impossible to achieve several decades ago. For the past decade several such communities in the Open Science Grid (OSG) and the European Grid Infrastructure (EGI) have been using GlideinWMS to run complex application workflows to effectively share computational resources over the grid. GlideinWMS is a pilot-based workload management system (WMS) that creates on demand, a dynamically sized overlay HTCondor batch system on grid resources. At present, the computational resources shared over the grid are just adequate to sustain the computing needs. We envision that the complexity of the science driven by "Big Data" will further push the need for computational resources. To fulfill their increasing demands and/or to run specialized workflows, some of the big communities like CMS are investigating the use of cloud computing as Infrastructure-As-A-Service (IAAS) with GlideinWMS as a potential alternative to fill the void. Similarly, communities with no previous access to computing resources can use GlideinWMS to setup up a batch system on the cloud infrastructure. To enable this, the architecture of GlideinWMS has been extended to enable support for interfacing GlideinWMS with different Scientific and commercial cloud providers like HLT, FutureGrid, FermiCloud and Amazon EC2. In this paper, we describe a solution for cloud bursting with GlideinWMS. The paper describes the approach, architectural changes and lessons learned while enabling support for cloud infrastructures in GlideinWMS.