RFS: efficient and flexible remote file access for MPI-IO
RFS: efficient and flexible remote file access for MPI-IO
复制标题
RFS：MPI-IO 高效灵活的远程文件访问
DOI：
10.1109/clustr.2004.1392604
复制
发表时间：
2004
期刊：
影响因子：
--
通讯作者：
M. Winslett
中科院分区：
文献类型：
--
作者：
Jonghyun Lee;R. Ross;R. Thakur;Xiaosong Ma;M. Winslett
科学应用程序经常需要访问远程文件系统。但是，由于网络速度慢和数据量大，远程I/O可能成为比本地I/O性能更严重的性能瓶颈。在这项工作中，我们提出了RFS，一个高性能的远程I/O设备，用于ROMIO，这是一个著名的MPI-IO实现。我们的简单、可移植和灵活的设计消除了以前远程I/O工作的缺点。特别是，RFS通过采用带有线程的主动缓冲（ABT）提高了远程I/O性能，它通过使用可用内存积极缓冲输出数据和在计算进行时使用线程执行后台I/O来隐藏I/O成本。我们的实验结果表明，带有ABT的RFS可以显著降低远程I/O可见成本，达到理论峰值吞吐量的92%。并发I/O活动导致的计算速度降低了0.2-6.2%，这与应用程序周转时间的整体性能改进相比相形见绌。
Scientific applications often need to access remote file systems. Because of slow networks and large data size, however, remote I/O can become an even more serious performance bottleneck than local I/O performance. In this work, we present RFS, a high-performance remote I/O facility for ROMIO, which is a well-known MPI-IO implementation. Our simple, portable, and flexible design eliminates the shortcomings of previous remote I/O efforts. In particular, RFS improves the remote I/O performance by adopting active buffering with threads (ABT), which hides I/O cost by aggressively buffering the output data using available memory and performing background I/O using threads while computation is taking place. Our experimental results show that RFS with ABT can significantly reduce the remote I/O visible cost, achieving up to 92% of the theoretical peak throughput. The computation slowdown caused by concurrent I/O activities was 0.2-6.2%, which is dwarfed by the overall performance improvement in application turnaround time.