DataStager: scalable data staging services for petascale applications
DataStager: scalable data staging services for petascale applications
复制标题
DOI：
10.1007/s10586-010-0135-6
复制
发表时间：
2009-06
期刊：
影响因子：
--
通讯作者：
H. Abbasi;M. Wolf;G. Eisenhauer;S. Klasky;K. Schwan;F. Zheng
中科院分区：
文献类型：
--
作者：
H. Abbasi;M. Wolf;G. Eisenhauer;S. Klasky;K. Schwan;F. Zheng
千万亿次机器的已知挑战是：（1）高性能应用程序的I/O成本可能很高，特别是对于像检查点这样的输出任务，以及（2）来自I/O动作的噪声可能会将不期望的延迟注入到单个计算节点上的此类代码的运行时中。本文介绍了灵活的“DataStager”框架，用于数据分级和替代服务，共同解决（1）和（2）。在存储之前将输出数据从计算节点移动到暂存或I/O节点的数据暂存服务用于减少应用程序总处理时间的I/O开销，并且数据暂存的显式管理在从千万亿次机器的计算分区提取输出数据时提供减少的扰动。在橡树岭国家实验室的Cray XT机器上的DataStager实验评估建立了智能数据分级的必要性和我们的方法的高性能，使用GTC融合建模代码和基准测试运行在1000+处理器。
Known challenges for petascale machines are that (1) the costs of I/O for high performance applications can be substantial, especially for output tasks like checkpointing, and (2) noise from I/O actions can inject undesirable delays into the runtimes of such codes on individual compute nodes. This paper introduces the flexible 'DataStager' framework for data staging and alternative services within that jointly address (1) and (2). Data staging services moving output data from compute nodes to staging or I/O nodes prior to storage are used to reduce I/O overheads on applications' total processing times, and explicit management of data staging offers reduced perturbation when extracting output data from a petascale machine's compute partition. Experimental evaluations of DataStager on the Cray XT machine at Oak Ridge National Laboratory establish both the necessity of intelligent data staging and the high performance of our approach, using the GTC fusion modeling code and benchmarks running on 1000+ processors.