Orchestrating Docker Containers in the HPC Environment
Orchestrating Docker Containers in the HPC Environment
复制标题
在 HPC 环境中编排 Docker 容器
DOI：
10.1007/978-3-319-20119-1_36
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Colin C. Venters
中科院分区：
文献类型：
--
作者：
Joshua Higgins;Violeta Holmes;Colin C. Venters
Linux容器技术已经证明了它在云计算中作为虚拟化的轻量级替代品的有用性，同时仍然提供了足够好的资源隔离。Docker正在成为管理Linux容器的流行运行时，提供管理工具和简单的文件格式。与传统虚拟机和裸机相比，对容器性能的研究表明，容器可以在处理，内存和网络吞吐量方面实现接近本地的速度。作为一项诞生于云端的技术，它正在进入科学计算领域，既作为共享实验应用程序的格式，也作为基于云的执行的范例。然而，它在传统的集群和网格计算中的应用还没有被探索。它提供了一个运行时环境，其中典型的集群和并行应用程序有机会以本机速度执行，同时与自己的特定（或遗留）库版本和支持软件捆绑在一起。这为集群和网格计算的致命弱点提供了一个解决方案，它要求用户对本地软件基础设施有深入的了解。使用Docker通过提供通用的定义格式和可重复的执行环境，使我们更接近集群内更有效的作业和资源管理。在本文中，我们介绍了在集群环境中部署Docker容器的工作结果，并评估了其作为高性能并行执行运行时的适用性。我们的研究结果表明，容器可以用于定制MPI应用程序的运行时环境，而不会影响性能，并为科学计算用户提供更好的服务质量。
Linux container technology has more than proved itself useful in cloud computing as a lightweight alternative to virtualisation, whilst still offering good enough resource isolation. Docker is emerging as a popular runtime for managing Linux containers, providing both management tools and a simple file format. Research into the performance of containers compared to traditional Virtual Machines and bare metal shows that containers can achieve near native speeds in processing, memory and network throughput. A technology born in the cloud, it is making inroads into scientific computing both as a format for sharing experimental applications and as a paradigm for cloud based execution. However, it has unexplored uses in traditional cluster and grid computing. It provides a run time environment in which there is an opportunity for typical cluster and parallel applications to execute at native speeds, whilst being bundled with their own specific (or legacy) library versions and support software. This offers a solution to the Achilles heel of cluster and grid computing that requires the user to hold intimate knowledge of the local software infrastructure. Using Docker brings us a step closer to more effective job and resource management within the cluster by providing both a common definition format and a repeatable execution environment. In this paper we present the results of our work in deploying Docker containers in the cluster environment and an evaluation of its suitability as a runtime for high performance parallel execution. Our findings suggest that containers can be used to tailor the run time environment for an MPI application without compromising performance, and would provide better Quality of Service for users of scientific computing.