Scalable Data Coupling Abstraction for Data-Intensive Simulation Workflows

数据密集型仿真工作流程的可扩展数据耦合抽象

• 批准号: 1310283
• 负责人: Manish Parashar
• 金额: $ 54.73万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-05-01 至 2017-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1310283&HistoricalAwards=false
• 关键词: Scalable; Data; Coupling; Abstraction; Intensive

A Scalable Data Management Abstraction for Large-scale Coupled Simulation WorkflowsCoupled scientific simulation workflows, integrating multiple physics and scales and running at very large scales on high-end resources, have the potential for achieving unprecedented levels of accuracy and providing dramatic insights into complex phenomena. However, the coupled component of these simulation workflows need to interact and exchange significant amounts of data at runtime, and the data often has to be transformed as it flows from source to destination. As the volumes and generation rates of this data grow, the costs (latencies and energy) associated with extracting this data and transporting it for coupling, transformation and analysis have become the dominating overheads and are dictating the level of performance and productivity that can be achieved.The goal of this project is to address these challenges and to develop conceptual solutions as well as a software framework that can enable the large-scale data-intensive simulations. Our approach is based on the premise that given the large data volumes and associated costs, data will have to be largely processed online, ?in-situ? and ?in-transit? while it is staged using resources within the computational platform, and the programming and runtime system must provide abstractions and mechanisms that facilitate such data processing. Our effort is organized around three key research thrusts: (1) Programming abstractions for in-situ/in-transit data management; (2) Design and implementation of a scalable data staging substrate; and (3) Data-centric mapping and scheduling.Data and compute intensive simulations are becoming increasingly critical to a wide range of science and engineering domains, and as a result, this research has the potential to drive research and innovations in these domains. The developed framework and benchmarks also provide computer scientists with a substrate to experiment with and explore data-centric research. The development of human resources, including the training of students, researchers and software professions, as well as outreach to minorities and underrepresented group, is integral to all aspects of this effort.

一个可扩展的数据管理抽象的大规模耦合仿真工作流耦合的科学仿真工作流，集成多个物理和规模和运行在非常大的规模上的高端资源，有可能实现前所未有的准确性水平，并提供戏剧性的见解复杂的现象。然而，这些仿真工作流的耦合组件需要在运行时交互和交换大量数据，并且数据通常必须在从源流向目的地时进行转换。随着这些数据的数量和生成速率的增长，与提取该数据并传输该数据以用于耦合相关联的能量和能量，转换和分析已经成为主要的管理费用，并决定了可以实现的性能和生产力水平。本项目的目标是解决这些挑战，并开发概念性解决方案以及软件框架，大规模数据密集型模拟。我们的方法是基于这样一个前提，即考虑到大量的数据和相关的成本，数据将不得不在很大程度上在线处理，？原地？然后呢？在途？而它是使用计算平台内的资源来分级的，并且编程和运行时系统必须提供便于这种数据处理的抽象和机制。我们的工作围绕三个关键的研究方向：（1）在现场/在途数据管理的编程抽象：（2）可扩展的数据分级基底的设计和实现;以及（3）以数据为中心的映射和调度。数据和计算密集型仿真对于广泛的科学和工程领域正变得越来越重要，并且因此，这项研究有可能推动这些领域的研究和创新。开发的框架和基准还为计算机科学家提供了一个实验和探索以数据为中心的研究的基础。开发人力资源，包括培训学生、研究人员和软件专业人员，以及与少数群体和代表性不足的群体进行外联，是这一努力的所有方面的组成部分。

项目成果

Persistent Data Staging Services for Data Intensive In-situ Scientific Workflows

适用于数据密集型原位科学工作流程的持久数据暂存服务

• DOI: 10.1145/2912152.2912157
• 发表时间: 2016
• 期刊: Proceedings of the ACM International Workshop on Data-Intensive Distributed Computing
• 作者: Romanus, Melissa;Klasky, Scott;Chang, Choong-Seock;Rodero, Ivan;Zhang, Fan;Jin, Tong;Sun, Qian;Bui, Hoang;Parashar, Manish;Choi, Jong
• 通讯作者: Choi, Jong

Leveraging Renewable Energy in Edge Clouds for Data Stream Analysis in IoT

• DOI: 10.1109/ccgrid.2017.92
• 发表时间: 2017-05
• 期刊: 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
• 作者: Yunbo Li;Anne-Cécile Orgerie;I. Rodero;M. Parashar;Jean-Marc Menaud