DC: Medium: Intelligent Data Placement in Support of Scientific Workflows

DC：中：支持科学工作流程的智能数据放置

• 批准号: 0905032
• 负责人: Ann Chervenak
• 金额: $ 81万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0905032&HistoricalAwards=false
• 关键词: DC; Medium; Intelligent; Data; Placement

Transformative research is conducted via computational analyses of large data sets in the terabyte and petabyte range. These analyses are often enabled by scientific workflows, which provide automation and efficient and reliable execution on campus and national cyberinfrastructure resources. Workflows face many issues related to data management such as locating input data, finding necessary storage co-located with computing capabilities, and efficiently staging data so that the computation progresses but storage resources do not fill up. Such data placement decisions need to be made within the context of individual workflows and across multiple concurrent workflows. Scientific collaborations also need to perform data placement operations to disseminate and replicate key data sets. Additional challenges arise when multiple scientific collaborations share cyberinfrastructure and compete for limited storage and compute resources. This project will explore the interplay between data management and computation management for these scenarios. The project will include the design of algorithms and methodologies that support large-scale data management for efficient workflow-based computations composed of individual analyses and workflow ensembles while preserving policies governing data storage and access. The algorithms will be evaluated regarding their impact on performance of synthetic and real-world workflows running in simulated and physical cyberinfrastructures. New approaches to data and computation management can potentially transform how scientific analyses are conducted at the petascale. Besides advancing computer science, this work will have direct impact on data and computation management for a range of scientific disciplines that manage large data sets and use them in complex analyses running on cyberinfrastructure.

通过对TB和PB级的大型数据集进行计算分析，进行变革性研究。这些分析通常由科学工作流程实现，这些工作流程在校园和国家网络基础设施资源上提供自动化和高效可靠的执行。工作流面临着许多与数据管理相关的问题，例如定位输入数据，找到与计算能力协同定位的必要存储，以及有效地暂存数据，以便计算进行但存储资源不会填满。这样的数据放置决策需要在单个工作流的上下文中以及跨多个并发工作流进行。科学合作还需要进行数据放置操作，以传播和复制关键数据集。当多个科学合作共享网络基础设施并争夺有限的存储和计算资源时，会出现其他挑战。这个项目将探索这些场景中数据管理和计算管理之间的相互作用。该项目将包括设计算法和方法，以支持大规模数据管理，进行基于工作流程的高效计算，包括单独分析和工作流程集合，同时保留数据存储和访问政策。将评估这些算法对在模拟和物理网络基础设施中运行的合成和真实世界工作流性能的影响。数据和计算管理的新方法可能会改变千万亿次科学分析的进行方式。除了推进计算机科学，这项工作还将对一系列科学学科的数据和计算管理产生直接影响，这些学科管理大型数据集并将其用于网络基础设施上运行的复杂分析。