Collaborative Research: HECURA: A New Semantic-Aware Metadata Organization for Improved File-System Performance and Functionality in High-End Computing

最后修改日期: 06/29/09 最后修改者: Almadena Y. Chtchelkanova 摘要 现有的基于分层目录树组织的数据存储系统无法满足具有数十亿文件的大规模 EB 级文件系统中指数级增长的数据集和日益复杂的元数据查询的可扩展性和功能性要求。该项目重点关注一种新的去中心化语义感知元数据组织，该组织利用文件元数据的语义来提高系统可扩展性、减少复杂数据查询的查询延迟并增强文件系统功能。该研究有四个主要组成部分：1）利用元数据语义相关性以可扩展的方式组织元数据；2）利用新元数据组织的语义和可扩展特性显着加快复杂查询速度并改进文件系统功能；3）充分利用新元数据组织的语义感知来优化存储系统设计，例如缓存、预取和重复数据删除；4）在大规模存储系统中实现新元数据组织、复杂查询功能和系统设计优化。该项目通过对存储系统研究的贡献以及与现有 NSF-REU 站点奖和 NSF-ITEST 奖的整合，对数据密集型科学和工程应用、研究生和本科生教育以及 K-12 教育产生更广泛的影响。

Last Modified Date: 06/29/09 Last Modified By: Almadena Y. Chtchelkanova Abstract Existing data storage systems based on the hierarchical directory-tree organization do not meet the scalability and functionality requirements for exponentially growing datasets and increasingly complex metadata queries in large-scale Exabyte-level file systems with billions of files. This project focuses on a new decentralized semantic-aware metadata organization that exploits semantics of file metadata to improve system scalability, reduce query latency for complex data queries, and enhance file system functionality. The research has four major components: 1) exploit metadata semantic-correlation to organize metadata in a scalable way, 2) exploit the semantic and scalable nature of the new metadata organization to significantly speed up complex queries and improve file system functionality, 3) fully leverage the semantic-awareness of the new metadata organization to optimize storage system designs, such as caching, prefetching, and data de-duplication, and 4) implement the new metadata organization, complex query functions, and system design optimizations in large-scale storage systems. This project has broader impact to data-intensive scientific and engineering applications, graduate and undergraduate education, and K-12 education through its contributions to storage system research and its integration with an existing NSF-REU site award and an NSF-ITEST award.