Scalable Data Coupling Abstraction for Data-Intensive Simulation Workflows

耦合的科学仿真工作流，集成了多种物理和规模，并在高端资源上以非常大的规模运行，有可能达到前所未有的精度水平，并提供对复杂现象的深刻见解。然而，这些模拟工作流的耦合组件需要在运行时交互和交换大量数据，并且数据在从源流向目的地时经常必须进行转换。随着数据量和生成速度的增长，与提取数据和传输数据以进行耦合、转换和分析相关的成本（延迟和能源）已经成为主要的开销，并决定了可以实现的性能和生产力水平。该项目的目标是解决这些挑战，并开发概念性解决方案以及能够实现大规模数据密集型模拟的软件框架。我们的方法是基于这样一个前提，即考虑到庞大的数据量和相关成本，数据必须大部分在线、就地处理。和在途的?虽然它是使用计算平台内的资源进行分阶段处理的，但是编程和运行时系统必须提供抽象和机制来促进这种数据处理。我们的工作围绕三个关键研究重点：(1)原位/在途数据管理的编程抽象；(2)设计和实现可扩展的数据分级基板；(3)以数据为中心的映射和调度。数据和计算密集型模拟在广泛的科学和工程领域变得越来越重要，因此，这项研究有可能推动这些领域的研究和创新。开发的框架和基准还为计算机科学家提供了实验和探索以数据为中心的研究的基础。人力资源的开发，包括培训学生、研究人员和软件专业人员，以及向少数民族和代表性不足的群体伸出援手，是这项努力所有方面的组成部分。

A Scalable Data Management Abstraction for Large-scale Coupled Simulation WorkflowsCoupled scientific simulation workflows, integrating multiple physics and scales and running at very large scales on high-end resources, have the potential for achieving unprecedented levels of accuracy and providing dramatic insights into complex phenomena. However, the coupled component of these simulation workflows need to interact and exchange significant amounts of data at runtime, and the data often has to be transformed as it flows from source to destination. As the volumes and generation rates of this data grow, the costs (latencies and energy) associated with extracting this data and transporting it for coupling, transformation and analysis have become the dominating overheads and are dictating the level of performance and productivity that can be achieved.The goal of this project is to address these challenges and to develop conceptual solutions as well as a software framework that can enable the large-scale data-intensive simulations. Our approach is based on the premise that given the large data volumes and associated costs, data will have to be largely processed online, ?in-situ? and ?in-transit? while it is staged using resources within the computational platform, and the programming and runtime system must provide abstractions and mechanisms that facilitate such data processing. Our effort is organized around three key research thrusts: (1) Programming abstractions for in-situ/in-transit data management; (2) Design and implementation of a scalable data staging substrate; and (3) Data-centric mapping and scheduling.Data and compute intensive simulations are becoming increasingly critical to a wide range of science and engineering domains, and as a result, this research has the potential to drive research and innovations in these domains. The developed framework and benchmarks also provide computer scientists with a substrate to experiment with and explore data-centric research. The development of human resources, including the training of students, researchers and software professions, as well as outreach to minorities and underrepresented group, is integral to all aspects of this effort.