Dynamic Traffic Control of Staging Traffic on the Interconnect of the HPC Cluster System
Dynamic Traffic Control of Staging Traffic on the Interconnect of the HPC Cluster System
复制标题
DOI：
10.1109/access.2020.3035158
复制
发表时间：
2020
期刊：
影响因子：
3.9
通讯作者：
Arata Endo;Hiroki Ohtsuji;Erika Hayashi;Eiji Yoshida;Chunghan Lee;S. Date;S. Shimojo
中科院分区：
文献类型：
--
作者：
Arata Endo;Hiroki Ohtsuji;Erika Hayashi;Eiji Yoshida;Chunghan Lee;S. Date;S. Shimojo
高性能计算（HPC）集群系统有时采用由本地和全局文件系统组成的两层文件系统来实现存储容量和性能。在这样的集群系统中，应用程序的输入数据需要从全局存储器暂存到本地存储器中，并且输出数据需要从本地存储器暂存到全局存储器。必须高效快速地执行此分段操作以获得更高的作业吞吐量，因为低效的分段操作会阻止执行等待的作业请求。具体地，在具有由存储节点和计算节点共享的超额预订互连的集群系统的情况下，节点间通信和该分级操作业务冲突，这可能降低作业吞吐量。在这项研究中，我们专注于流量碰撞的节点间通信和分期流量，以提高作业吞吐量，针对的集群系统与超额认购互连这两种类型的流量流动。换句话说，是否来自分级操作的业务流的动态控制导致的作业吞吐量的改善或不进行调查。对于调查，我们提出了一个交通冲突避免方法，动态配置一组数据路径的每一种类型的流量，而分级操作进行。在这篇文章中的评估表明，所提出的方法避免了交通冲突，并加快了22.0%，我们的集群系统的分期操作。此外，该评估表明所提出的方法所引起的应用程序的开销是可以忽略不计的。此外，8.7%的作业执行时间减少所提出的方法。
High-performance computing (HPC) cluster systems sometimes adopt a two-layered file system composed of local and global file systems to achieve both capacity and performance in storage. In such a cluster system, the input data of an application needs to be staged from the global storage into the local storage, and the output data needs to be staged from the local storage out to the global storage. This staging operation must be efficiently and quickly performed to gain higher job throughput because an inefficient staging operation prevents waiting job requests from being executed. In particular, in the case of the cluster system with the oversubscribed interconnect shared by the storage and the computing nodes, the inter-node communication and this staging operation traffic collides, which may degrade the job throughput. In this research, we focus on the traffic collision of the inter-node communication and the staging traffic to improve job throughput, targeting the cluster system with the oversubscribed interconnect where these two types of traffic flow. In other words, whether the dynamic control of the traffic flow derived from the staging operation leads to the improvement in the job throughput or not is investigated. For the investigation, we present a traffic collision avoidance method to dynamically configure a set of data paths for each type of the traffic only while the staging operation is conducted. The evaluation in this article shows that the proposed method avoids a traffic collision and accelerates the staging operation by 22.0% on our cluster system. Also, this evaluation indicates the overhead of the application incurred by the proposed method is negligible. Furthermore, 8.7% of the job execution time is reduced by the proposed method.