Collaborative Research: Scalable Data Management Using Metadata and Provenance

协作研究：使用元数据和来源的可扩展数据管理

• 批准号: 0937914
• 负责人: Margo Seltzer
• 金额: $ 35.16万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-10-01 至 2012-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0937914&HistoricalAwards=false
• 关键词: Collaborative; Research; Scalable; Data; Management

Abstract This project is developing new techniques for identifying and managing files, replacing tree-structured file names with content- and metadata- based search access. By leveraging existing work in search and recognizing the explosion in the volume of data stored, this project enables users to find and access their data in natural and intuitive ways, based on the files' contents, tags the user has assigned, system metadata, and provenance (information about the file's origins). This research targets high-end computing (HEC) users, who manage billions of files generated by measurement devices, experimentation, or scientific workflows. The techniques and system developed are also applicable to general-purpose computing. Realizing this goal requires advances in several areas. First, the project is designing and developing fast, scalable mechanisms to gather, maintain and index the large volume of metadata and provenance that HEC applications and users generate. This project is also exploring search algorithms that operate on graph structures, enabling users to find files "near" their current workspace. To enable users to access this functionality, the project is developing a new "language" that facilitates the kind of searches that users need.

这个项目正在开发识别和管理文件的新技术，用基于内容和元数据的搜索访问取代树形结构的文件名。通过利用现有的搜索工作并认识到存储的数据量的爆炸性增长，该项目使用户能够以自然和直观的方式查找和访问他们的数据，基于文件的内容，用户分配的标签，系统元数据和出处（有关文件来源的信息）。该研究针对高端计算（HEC）用户，他们管理由测量设备、实验或科学工作流程生成的数十亿个文件。所开发的技术和系统也适用于通用计算。实现这一目标需要在若干领域取得进展。首先，该项目正在设计和开发快速、可扩展的机制，以收集、维护和索引HEC应用程序和用户生成的大量元数据和出处。该项目还在探索在图结构上操作的搜索算法，使用户能够在其当前工作区”附近”找到文件。为了使用户能够访问这一功能，该项目正在开发一种新的“语言”，以方便用户需要的搜索类型。