Collaborative Research: HECURA: A New Semantic-Aware Metadata Organization for Improved File-System Performance and Functionality in High-End Computing

现有的基于层次目录树组织的数据存储系统不能满足具有数十亿个文件的大规模EB级文件系统中指数增长的数据集和日益复杂的元数据查询的可扩展性和功能要求。这个项目的重点是一个新的分散的语义感知元数据组织，利用文件元数据的语义来提高系统的可扩展性，减少复杂数据查询的查询延迟，并增强文件系统的功能。该研究有四个主要组成部分：1）利用元数据语义相关性来以可伸缩的方式组织元数据，2）利用新的元数据组织的语义和可伸缩性质来显著加速复杂查询并改进文件系统功能，3）充分利用新的元数据组织的语义感知来优化存储系统设计，诸如高速缓存、预取和重复数据删除，4）在大规模存储系统中实现新的元数据组织、复杂查询功能和系统设计优化。 该项目通过对存储系统研究的贡献以及与现有NSF-REU站点奖和NSF-ITEST奖的集成，对数据密集型科学和工程应用、研究生和本科生教育以及K-12教育产生了更广泛的影响。

Existing data storage systems based on the hierarchical directory-tree organization do not meet the scalability and functionality requirements for exponentially growing datasets and increasingly complex metadata queries in large-scale Exabyte-level file systems with billions of files. This project focuses on a new decentralized semantic-aware metadata organization that exploits semantics of file metadata to improve system scalability, reduce query latency for complex data queries, and enhance file system functionality. The research has four major components: 1) exploit metadata semantic-correlation to organize metadata in a scalable way, 2) exploit the semantic and scalable nature of the new metadata organization to significantly speed up complex queries and improve file system functionality, 3) fully leverage the semantic-awareness of the new metadata organization to optimize storage system designs, such as caching, prefetching, and data de-duplication, and 4) implement the new metadata organization, complex query functions, and system design optimizations in large-scale storage systems. This project has broader impact to data-intensive scientific and engineering applications, graduate and undergraduate education, and K-12 education through its contributions to storage system research and its integration with an existing NSF-REU site award and an NSF-ITEST award.