Quality of Service Guarantee for Scalable Parallel Storage Systems

本项目旨在设计、实现和评估一种可扩展的计算集群并行存储系统Platypus，当多个应用程序同时运行在同一集群上时，为单个应用程序提供带宽保证，并支持解耦的文件提取机制，可以有效地重叠CPU处理和磁盘访问，降低磁盘访问流的突发程度。Platypus的磁盘QoS保证算法在并发运行的并行应用之间提供了长期/短期的性能隔离，并通过利用QoS执行过程中的松弛来最大化磁盘的整体利用率。Platypus的文件预取机制应用了解耦架构的概念，该概念最初是为了弥合CPU和内存之间的差距而提出的，以实现接近完美的磁盘预取，并且可以有效地掩盖与并行应用程序中文件访问相关的磁盘I/O和网络延迟。为了评估Platypus的有效性和效率，pi建议构建一个并行文件访问跟踪播放器，使研究人员能够以可扩展的方式基于预收集的跟踪来评估并行文件系统。

This project aims to design, implement and evaluate a scalable parallel storage system for compute clusters called Platypus, that provides bandwidth guarantee for individual applications when multiple of them run on the same cluster simultaneously, and supports a decoupled file fetching mechanism that can effectively overlap CPU processing and disk access and decrease the degree of burstiness in disk access streams. Platypus's disk QoS guarantee algorithm offers long-term/short-term performance isolation among concurrently running parallel applications, and maximizes the overall disk utilization efficiency by exploiting the slack in the QoS enforcement process. Platypus's file prefetching mechanism applies the concept of decoupled architecture, which was originally proposed to bridge the gap between CPU and memory, to achieve close to perfect disk prefetching, and can effectively mask both disk I/O and networkingdelay associated with file accesses in for parallel applications. To evaluate the effectiveness and efficiency of Platypus, the PIs propose to build a parallel file access trace player which allows researchers to evaluate parallel file systems based on pre-collected traces in a scalable way.