Collective buffering: Improving parallel I/O performance
Collective buffering: Improving parallel I/O performance
复制标题
集体缓冲：提高并行 I/O 性能
DOI：
--
复制
发表时间：
1997
期刊：
影响因子：
--
通讯作者：
V. Lo
中科院分区：
文献类型：
--
作者：
B. Nitzberg;V. Lo
“并行I/O”是对在多个节点上运行的单个并行应用程序的支持;应用程序数据分布在节点之间，并被读取或写入单个逻辑文件，该文件本身分布在节点和磁盘上。并行I/O是从节点内存中的数据布局到磁盘上的文件布局的映射问题。由于映射可能非常复杂，并且涉及大量的数据移动，因此优化映射对于性能至关重要。我们讨论了我们的一般模型的问题，描述了我们设计的四个集体缓冲算法，并报告实验测试他们的性能在英特尔Paragon和IBM SP2都设在美国宇航局艾姆斯研究中心。我们的实验表明，标准技术的改进高达两个数量级，并有可能提供最低的硬件支持的峰值性能。
"Parallel I/O" is the support of a single parallel application run on many nodes; application data is distributed among the nodes, and is read or written to a single logical file, itself spread across nodes and disks. Parallel I/O is a mapping problem from the data layout in node memory to the file layout on disks. Since the mapping can be quite complicated and involve significant data movement, optimizing the mapping is critical for performance. We discuss our general model of the problem, describe four Collective Buffering algorithms we designed, and report experiments testing their performance on an Intel Paragon and an IBM SP2 both housed at NASA Ames Research Center. Our experiments show improvements of up to two order of magnitude over standard techniques and the potential to deliver peak performance with minimal hardware support.