HECURA: A Dynamic Application-specific I/O Architecture for High End Computing

高端计算机器上的磁盘I/O仍然是一个重要的性能瓶颈。为了提高并行I/O性能，开发了并行文件系统。然而，这些方法中的大多数都依赖于应用程序，它们的性能因应用程序而异。通过更好地理解客户端和文件服务器端的I/O访问特征，可以提高并行I/O的性能。为了满足高端计算的需求，迫切需要研究下一代智能和特定于应用程序的I/O体系结构。我们提出了一个动态的特定于应用程序的I/O架构，该架构根据应用程序的I/O特征定制各种并行I/O优化。这种体系结构是动态的，因为它的底层优化策略能够适应不同应用程序中的变化，以获得最佳性能。提出的研究有两个方面：1)理解I/O行为，2)开发特定于应用程序的数据布局、预取和缓存优化，以形成集成的特定于应用程序的I/O体系结构。已经确定了几个技术障碍，包括I/O访问签名，编译器分析，全局感知协调缓存，集体预取，数据布局优化和分布策略。在PVFS2并行文件系统下对这些新提出的解决方案和技术进行了测试，并给出了详细的方案。

Disk I/O on high-end computing machines continues to be a significant performance bottleneck. Parallel file systems have been developed to improve parallel I/O performance. However, most of these methods are application dependent and their performance varies largely from application to application. The performance of parallel I/O can be improved with better understanding of I/O access characteristics at both client and file-server side. There is a great need for research into next-generation intelligent and application-specific I/O architectures to meet the demand of highend computing.We propose a dynamic application-specific I/O architecture that tailors various parallel I/O optimizations based on I/O characteristics of applications. This architecture is dynamic in the sense that its underlying optimization strategies are able to adapt to the variations in different applications for best performance. The proposed research is twofold: 1) understanding I/O behavior, 2) developing application-specific optimizations for data layout, prefetching, and caching to form an integrated application-specific I/O architecture. Several technical hurdles have been identified, which include I/O access signature, compiler analysis, global-aware coordinatedcaching, collective prefetching, data layout optimization and distribution strategies. Solutions are proposed and detailed plans are provided to test these newly proposed solutions and techniques under the PVFS2 parallel file system.