Distributed MPI cluster with Docker Swarm mode
Distributed MPI cluster with Docker Swarm mode
复制标题
采用 Docker Swarm 模式的分布式 MPI 集群
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Doina Bein
中科院分区：
文献类型：
--
作者：
Nikyle Nguyen;Doina Bein
MPI是一种在高性能计算环境中得到广泛应用的成熟技术。但是，设置MPI集群可能具有挑战性且耗时。本文通过使用现代容器化技术（Docker）和容器编排技术（Docker Swarm模式）来解决这一挑战，以自动化MPI集群的设置和部署。我们创建了一个即用型解决方案，用于在多台机器上运行的Docker容器集群中开发和部署MPI程序，使用Docker Swarm模式进行编排，以执行高计算任务。我们解释了创建将被实例化为MPI节点的Docker镜像时的注意事项，并描述了将完全连接的MPI集群设置为在Docker Swarm模式下运行的Docker容器所需的步骤。我们的目标是给出我们解决方案背后的基本原理，以便其他人可以适应不同的系统需求。所有预构建的Docker镜像、源代码、文档和屏幕截图都是公开的。
MPI is a well-established technology that is used widely in high-performance computing environment. However, setting up an MPI cluster can be challenging and time-consuming. This paper tackles this challenge by using modern containerization technology, which is Docker, and container orchestration technology, which is Docker Swarm mode, to automate the MPI cluster setup and deployment. We created a ready-to-use solution for developing and deploying MPI programs in a cluster of Docker containers running on multiple machines, orchestrated with Docker Swarm mode, to perform high computation tasks. We explain the considerations when creating Docker image that will be instantiated as MPI nodes, and we describe the steps needed to set up a fully connected MPI cluster as Docker containers running in a Docker Swarm mode. Our goal is to give the rationale behind our solution so that others can adapt to different system requirements. All pre-built Docker images, source code, documentation, and screencasts are publicly available.