HPC Cloud Architecture to Reduce HPC Workflow Complexity in Containerized Environments
HPC Cloud Architecture to Reduce HPC Workflow Complexity in Containerized Environments
复制标题
HPC 云架构可降低容器化环境中 HPC 工作流程的复杂性
作者：
Guohua Li;J. Woo;S. Lim
高性能计算（HPC）工作流程的复杂性是大多数国家超级计算中心提供HPC云服务的一个重要问题。这种复杂性问题尤为关键，因为它影响HPC资源的可扩展性、管理效率和使用便利性。为了解决这个问题，在发挥裸机级高性能优势的同时，基于容器的云解决方案应运而生。然而，仍然存在各种问题，例如HPC和云之间的隔离环境、安全问题、工作负载管理问题。我们提出了一种通过使用 Docker 和 Singularity（HPC 云领域最常用的容器平台）来降低这种复杂性的架构。该 HPC 云架构集成了图像管理和作业管理，这是 HPC 云工作流程的两个主要元素。为了评估所提出的架构的可服务性和性能，我们在 HPC 集群实验中开发并实现了一个平台。实验结果表明，所提出的HPC云架构可以降低复杂性，提供超级计算资源可扩展性、高性能、用户便利性、各种HPC应用和管理效率。
The complexity of high-performance computing (HPC) workflows is an important issue in the provision of HPC cloud services in most national supercomputing centers. This complexity problem is especially critical because it affects HPC resource scalability, management efficiency, and convenience of use. To solve this problem, while exploiting the advantage of bare-metal-level high performance, container-based cloud solutions have been developed. However, various problems still exist, such as an isolated environment between HPC and the cloud, security issues, and workload management issues. We propose an architecture that reduces this complexity by using Docker and Singularity, which are the container platforms most often used in the HPC cloud field. This HPC cloud architecture integrates both image management and job management, which are the two main elements of HPC cloud workflows. To evaluate the serviceability and performance of the proposed architecture, we developed and implemented a platform in an HPC cluster experiment. Experimental results indicated that the proposed HPC cloud architecture can reduce complexity to provide supercomputing resource scalability, high performance, user convenience, various HPC applications, and management efficiency.