VC3: A Virtual Cluster Service for Community Computation
VC3: A Virtual Cluster Service for Community Computation
复制标题
VC3：社区计算的虚拟集群服务
DOI：
10.1145/3219104.3219125
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
D. Thain
中科院分区：
文献类型：
--
作者：
Lincoln Bryant;Jeremy Van;B. Riedel;R. Gardner;J. C. Bejar;J. Hover;Benjamín Tovar;Kenyi Hurtado;D. Thain
传统的HPC计算设施提供大量的计算能力，但具有为满足本地需求而设计的固定环境。这使得用户部署跨多个站点的复杂应用程序非常具有挑战性，并且需要特定的应用程序软件、调度中间件或共享策略。由美国能源部资助的VC3项目旨在解决这些挑战，使研究人员能够轻松地聚合和共享资源，安装自定义软件环境，并通过“虚拟集群”的概念在多个HPC设施中部署集群框架。本文介绍了我们的原型自助服务VC3平台的设计，实施和初步经验，该平台自动部署集群框架在不同的计算设施。为了创建虚拟集群，VC3平台在安全的私有云中实现自定义头节点，指定调度中间件的选择，然后从远程设施分配资源，其中所需的软件和集群框架安装在用户空间中。随着资源从各个集群的预定节点变得可用，研究团队只需看到一个他们可以直接访问或与合作者共享的私有集群，例如科学网关社区。我们讨论了如何使用这项服务，需要共享资源，特定的中间件框架，复杂的应用程序和工作流程，在天体物理学，生物信息学和高能物理领域的研究合作。
A traditional HPC computing facility provides a large amount of computing power but has a fixed environment designed to satisfy local needs. This makes it very challenging for users to deploy complex applications that span multiple sites and require specific application software, scheduling middleware, or sharing policies. The DOE-funded VC3 project aims to address these challenges by making it possible for researchers to easily aggregate and share resources, install custom software environments, and deploy clustering frameworks across multiple HPC facilities through the concept of "virtual clusters". This paper presents the design, implementation, and initial experience with our prototype self-service VC3 platform which automates deployment of cluster frameworks across diverse computing facilities. To create a virtual cluster, the VC3 platform materializes a custom head node in a secure private cloud, specifies a choice of scheduling middleware, then allocates resources from the remote facilities where the desired software and clustering framework is installed in user space. As resources become available from scheduled nodes from individual clusters, the research team simply sees a private cluster they can access directly or share with collaborators, such as a science gateway community. We discuss how this service can be used by research collaborations requiring shared resources, specific middleware frameworks, and complex applications and workflows in the areas of astrophysics, bioinformatics and high energy physics.
集群上科学应用程序的自动依赖性管理
DOI：
10.1109/ic2e.2018.00026
发表时间：
2018
期刊：
IEEE International Conference on Cloud Engineering (IC2E
影响因子：
--
作者：
Tovar, Benjamin;Hazekamp, Nicholas;Kremer-Herman, Nathaniel;Thain, Douglas
通讯作者：
Thain, Douglas
影响因子：
7
作者：
Cantarel, Brandi L.;Korf, Ian;Yandell, Mark
通讯作者：
Yandell, Mark
rvGAHP：使用反向 SSH 连接的基于推送的作业提交
DOI：
10.1145/3150994.3151003
发表时间：
2017
期刊：
Proceedings of the 12th Workshop on Workflows in Support of Large-Scale Science
影响因子：
--
作者：
Callaghan, Scott;Juve, Gideon;Vahi, Karan;Maechling, Philip J.;Jordan, Thomas H.;Deelman, Ewa
通讯作者：
Deelman, Ewa