HECURA: A Dynamic Application-specific I/O Architecture for High End Computing

高端计算机上的磁盘I/O仍然是一个重要的性能瓶颈。为了提高并行I/O性能，已经开发了并行文件系统。然而，这些方法中的大多数都依赖于应用程序，并且它们的性能因应用程序的不同而有很大差异。通过更好地了解客户端和文件服务器端的I/O访问特征，可以提高并行I/O的性能。为了满足高端计算的需求，迫切需要研究新一代智能和特定于应用的I/O体系结构，提出了一种动态的特定于应用的I/O体系结构，该体系结构根据应用的I/O特性定制各种并行I/O优化。该体系结构是动态的，因为它的底层优化策略能够适应不同应用程序的变化，以获得最佳性能。建议的研究包括两个方面：1)了解I/O行为；2)针对数据布局、预取和缓存开发特定于应用程序的优化，以形成集成的特定于应用程序的I/O体系结构。已经确定了几个技术障碍，其中包括I/O访问签名、编译器分析、全球感知的协调缓存、集体预取、数据布局优化和分布策略。提出了解决方案，并给出了在PVFS2并行文件系统下对这些新提出的解决方案和技术进行测试的详细计划。

Disk I/O on high-end computing machines continues to be a significant performance bottleneck. Parallel file systems have been developed to improve parallel I/O performance. However, most of these methods are application dependent and their performance varies largely from application to application. The performance of parallel I/O can be improved with better understanding of I/O access characteristics at both client and file-server side. There is a great need for research into next-generation intelligent and application-specific I/O architectures to meet the demand of highend computing.We propose a dynamic application-specific I/O architecture that tailors various parallel I/O optimizations based on I/O characteristics of applications. This architecture is dynamic in the sense that its underlying optimization strategies are able to adapt to the variations in different applications for best performance. The proposed research is twofold: 1) understanding I/O behavior, 2) developing application-specific optimizations for data layout, prefetching, and caching to form an integrated application-specific I/O architecture. Several technical hurdles have been identified, which include I/O access signature, compiler analysis, global-aware coordinatedcaching, collective prefetching, data layout optimization and distribution strategies. Solutions are proposed and detailed plans are provided to test these newly proposed solutions and techniques under the PVFS2 parallel file system.