DataSpaces: an interaction and coordination framework for coupled simulation workflows
DataSpaces: an interaction and coordination framework for coupled simulation workflows
复制标题
DOI：
10.1007/s10586-011-0162-y
复制
发表时间：
2012-06-01
影响因子：
4.4
通讯作者：
Klasky, Scott
中科院分区：
文献类型：
--
作者：
Docan, Ciprian;Parashar, Manish;Klasky, Scott
新兴的高性能分布式计算环境正在实现科学和工程中的新端到端公式，涉及多个交互过程和数据密集型应用程序工作流程。例如，当前的融合模拟工作正在探索同时模拟单独的应用程序过程（例如核心和边缘湍流）的耦合模型和代码。这些组件在不同的高性能计算资源上运行，需要在运行时进行交互，以及用于数据监视，数据分析和可视化以及数据归档的服务。结果，它们需要有效且可扩展的支持，以动态和灵活的耦合和互动，这仍然是一个挑战。本文介绍了数据公告，一种灵活的互动和协调基板，该基材解决了这一挑战。数据公告本质上实现了语义专业的虚拟虚拟共享空间抽象，该抽象可以由应用程序工作流中的所有组件和服务共同访问。它使实时数据可以从运行的仿真组件中提取，在线索引此数据，然后允许使用其他组件和服务通过使用语义意义的运算符对其进行监视，查询和访问。潜在的数据传输是异步，低空的，并且在很大程度上是内存到记忆的。提出了使用耦合融合仿真工作流对数据源的设计，实现和实验评估。
Emerging high-performance distributed computing environments are enabling new end-to-end formulations in science and engineering that involve multiple interacting processes and data-intensive application workflows. For example, current fusion simulation efforts are exploring coupled models and codes that simultaneously simulate separate application processes, such as the core and the edge turbulence. These components run on different high performance computing resources, need to interact at runtime with each other and with services for data monitoring, data analysis and visualization, and data archiving. As a result, they require efficient and scalable support for dynamic and flexible couplings and interactions, which remains a challenge. This paper presents DataSpaces a flexible interaction and coordination substrate that addresses this challenge. DataSpaces essentially implements a semantically specialized virtual shared space abstraction that can be associatively accessed by all components and services in the application workflow. It enables live data to be extracted from running simulation components, indexes this data online, and then allows it to be monitored, queried and accessed by other components and services via the space using semantically meaningful operators. The underlying data transport is asynchronous, low-overhead and largely memory-to-memory. The design, implementation, and experimental evaluation of DataSpaces using a coupled fusion simulation workflow is presented.