Efficient Asynchronous I/O with Request Merging
Efficient Asynchronous I/O with Request Merging
复制标题
DOI：
10.1109/ipdpsw59300.2023.00107
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Md. Kamal Hossain Chowdhury;Houjun Tang;J. L. Bez;P. Bangalore;S. Byna
中科院分区：
文献类型：
--
作者：
Md. Kamal Hossain Chowdhury;Houjun Tang;J. L. Bez;P. Bangalore;S. Byna
随着百亿亿次计算的进步，科学数据量日益增加。有效的数据访问对于科学发现是必要的。不幸的是，I/O 性能没有得到改善，就像 CPU 和网络速度一样。因此，I/O 操作比数据生成或分析花费更长的时间。异步 I/O 已被提议通过重叠 I/O 和计算时间来减轻 I/O 瓶颈。然而，多个小型写入操作可能会削弱异步 I/O 的优势，因为 I/O 时间明显长于计算时间，几乎没有时间重叠。为了克服这些问题，我们提出了一种优化技术来合并小型连续写入操作。我们将我们的解决方案集成到 HDF5 异步 I/O VOL 连接器中，并演示了自动、透明地合并 HDF5 写入操作的有效性，而无需对应用程序进行任何代码更改。
With the advancement of exascale computing, the amount of scientific data is increasing day by day. Efficient data access is necessary for scientific discoveries. Unfortunately, the I/O performance is not improved, like the CPU and network speed. So, I/O operations take longer time than data generation or analysis. Asynchronous I/O has been proposed to extenuate the I/O bottleneck by overlapping I/O and computation time. However, multiple small write operations can diminish the benefits of asynchronous I/O, as the I/O time becomes significantly longer than the compute time, with little time to overlap with. To overcome these issues, we present an optimization technique to merge small contiguous write operations. We integrated our solution into the HDF5 asynchronous I/O VOL connector and demonstrated the effectiveness of merging HDF5 write operations automatically and transparently without requiring any code change from the application.