On Overlapping Communication and File I/O in Collective Write Operation
On Overlapping Communication and File I/O in Collective Write Operation
复制标题
DOI：
10.1109/ipdpsw50202.2020.00175
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Raafat Feki;E. Gabriel
中科院分区：
文献类型：
--
作者：
Raafat Feki;E. Gabriel
许多并行科学应用程序花费大量时间阅读和写入数据文件。集体I/O操作允许通过跨进程重新分布数据以匹配文件系统上的数据布局来优化进程组的文件访问。在大多数并行I/O库中，集体I/O操作的实现基于两阶段I/O算法，该算法由通信阶段和文件访问阶段组成。本文评估了两阶段I/O算法重叠两个内部周期的各种设计方案，并探讨了在洗牌阶段使用不同的数据传输原语，包括非阻塞双侧通信和单侧通信的多个版本。结果表明，重叠算法结合异步I/O优于重叠的方法，只依赖于非阻塞通信。然而，在绝大多数测试用例中，单边通信并没有导致双边通信的性能改善。
Many parallel scientific applications spend a significant amount of time reading and writing data files. Collective I/O operations allow to optimize the file access of a process group by redistributing data across processes to match the data layout on the file system. In most parallel I/O libraries, the implementation of collective I/O operations is based on the two-phase I/O algorithm, which consists of a communication phase and a file access phase. This papers evaluates various design options for overlapping two internal cycles of the two-phase I/O algorithm, and explores using different data transfer primitives for the shuffle phase, including non-blocking two-sided communication and multiple versions of one-sided communication. The results indicate that overlap algorithms incorporating asynchronous I/O outperform overlapping approaches that only rely on nonblocking communication. However, in the vast majority of the testcases one-sided communication did not lead to performance improvements over two-sided communication.