One optimized I/O configuration per HPC application: leveraging the configurability of cloud
One optimized I/O configuration per HPC application: leveraging the configurability of cloud
复制标题
DOI：
10.1145/2103799.2103818
复制
发表时间：
2011-07
期刊：
影响因子：
--
通讯作者：
Mingliang Liu;Jidong Zhai;Yan Zhai;Xiaosong Ma;Wenguang Chen
中科院分区：
文献类型：
--
作者：
Mingliang Liu;Jidong Zhai;Yan Zhai;Xiaosong Ma;Wenguang Chen
将HPC（高性能计算）应用程序迁移到云平台（如Amazon EC2群集计算平台（CCI））是一种趋势。虽然现有的研究主要集中在虚拟化环境和互连技术对并行程序的性能影响上，但我们认为云支持的可配置性是另一个需要探索的重要方面。与传统HPC平台不同，在云驻留虚拟群集上，可以轻松更改I/O配置，例如文件系统的选择、I/O节点的数量和虚拟磁盘的类型，以满足不同应用程序的I/O要求。在本文中，我们讨论了如何使用云平台来为单个I/O密集型MPI应用程序形成定制和平衡的I/O子系统。通过我们的初步评估，我们证明，不同的应用程序将受益于个性化定制的I/O系统配置。对于给定的I/O密集型应用程序，不同的I/O设置可能会导致显著的整体应用程序性能或成本差异（高达2.5倍）。我们的探索表明，在云中的HPC应用程序的自定义系统配置是重要的和不平凡的。
There is a trend to migrate HPC (High Performance Computing) applications to cloud platforms, such as the Amazon EC2 Cluster Compute Instances (CCIs). While existing research has mainly focused on the performance impact of virtualized environments and interconnect technologies on parallel programs, we suggest that the configurability enabled by clouds is another important dimension to explore. Unlike on traditional HPC platforms, on a cloud-resident virtual cluster it is easy to change the I/O configurations, such as the choice of file systems, the number of I/O nodes, and the types of virtual disks, to fit the I/O requirements of different applications. In this paper, we discuss how cloud platforms can be employed to form customized and balanced I/O subsystems for individual I/O-intensive MPI applications. Through our preliminary evaluation, we demonstrate that different applications will benefit from individually tailored I/O system configurations. For a given I/O-intensive application, different I/O settings may lead to significant overall application performance or cost difference (up to 2.5-fold). Our exploration indicates that customized system configuration for HPC applications in the cloud is important and non-trivial.