Swift/T: Large-Scale Application Composition via Distributed-Memory Dataflow Processing
Swift/T: Large-Scale Application Composition via Distributed-Memory Dataflow Processing
复制标题
Swift/T：通过分布式内存数据流处理构建大规模应用程序
DOI：
10.1109/ccgrid.2013.99
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Ian T Foster
中科院分区：
文献类型：
--
作者：
J. Wozniak;Timothy G. Armstrong;M. Wilde;D. Katz;E. Lusk;Ian T Foster
从概念上讲，许多科学应用都是由独立的组件任务（如参数研究、优化或其他搜索）构建而成的。这些任务的大量批处理可以在高端计算系统上执行，但是，如何协调独立进程、它们的数据以及它们的数据依赖关系是一个重大的可扩展性挑战。必须解决许多问题，包括负载平衡、数据分布、通知、并发编程以及与现有代码的链接。在这项工作中，我们介绍了 Swift/T，这是一种编程语言和运行时，可以快速开发高度并发、任务并行的应用程序。Swift/T 由几项应对可扩展性挑战的使能技术组成，为用户编程和调试提供了高级优化编译器，并提供了将 C/C++/Fortran 中的用户代码绑定到逻辑脚本中的工具。在这项工作中，我们介绍了 Swift/T 解决方案，并展示了 IBM Blue Gene/Pand Blue Gene/Q 的扩展结果。
Many scientific applications are conceptually built up from independent component tasks as a parameter study, optimization, or other search. Large batches of these tasks may be executed on high-end computing systems, however, the coordination of the independent processes, their data, and their data dependencies is a significant scalability challenge. Many problems must be addressed, including load balancing, data distribution, notifications, concurrent programming, and linking to existing codes. In this work, we present Swift/T, a programming language and runtime that enables the rapid development of highly concurrent, task-parallel applications. Swift/Tis composed of several enabling technologies to address scalability challenges, offers a high-level optimizing compiler for user programming and debugging, and provides tools for binding user code in C/C++/Fortran into a logical script. In this work, we describe the Swift/T solution and present scaling results from the IBM Blue Gene/Pand Blue Gene/Q.