Collaborative Research: SI2-SSI: Swift/E: Integrating Parallel Scripted Workflow into the Scientific Software Ecosystem

科学和工程研究越来越依赖于重复执行一系列复杂的步骤(即工作流程)来形成假设、进行实验、分析结果和完善理论。计算在整个工作流程中通常是必不可少的，在这种情况下，软件可以通过管理计算和数据工作流程来提高工作效率。SWIFT就是这样一种开源工作流系统，已经开发并广泛应用于从材料模拟和气候建模到神经科学和基因组学的不同领域。该项目通过将SWIFT与其他支持协作、可用性、可维护性和工作效率的软件系统集成，扩展了SWIFT的功能。新的生态系统SWIFT/E将使科学家和工程师能够更高效地创建和运行更大规模的计算工作流活动，并比以往更快、更容易地调试、执行、调整和传播这些活动。这些工作流程体现并传达了特定于科学研究每个领域的计算方法。SWIFT/E通过研究、教育和软件传播的集成计划，为大型用户社区实现社区参与和广泛的生产力优势。该项目通过创建构建和共享可重复使用的工作流库的实践模式，并通过培训学生、教育工作者和研究人员使用它们，来参与并服务于科学和工程社区。为了促进下一代受过计算培训的科学家的教育，SWIFT/E支持一个由NSF支持的“e-Labs”网络，在高中和本科水平教授协作并行计算科学的概念，每年接触到1000多名学生。开源的SWIFT/E“生态系统”将SWIFT与几个在当今国家和全球网络基础设施中发挥重要作用的科学软件元素相结合。这些元素是：用于科学工作流并行脚本编写的SWIFT；用于数据编目、管理和高速广域传输的Globus；用于工作流合成、执行和协作共享的基于Web的Galaxy工作流门户；用于交互开发、测试、调试和汇编高级编程和工作流语言的Jupyter；用于高效表达高级计算逻辑的Python和R；以及用于修订控制、代码传播和共享以及开发人员协作参与的“Git”和相关工具和Web门户。SWIFT的隐式并行编程语言是最小和紧凑的。SWIFT提供了一种工具，可以将其他脚本语言(目前是Python、R、Julia和TCL)嵌入其运行时环境。该项目将较新的极端规模“SWIFT/T”能力与灵活可移植的原始“SWIFT/K”版本相结合，使核心SWIFT/E软件元素更加强大和灵活，同时降低其持续支持成本。SWIFT/E通过扩展SWIFT的故障排除和语言间集成设施来增强可用性。SWIFT/E项目通过增强和创新的工作流程共享档案、新的培训材料和持续的用户支持计划以及自我维持和扩大的社区参与，吸引、支持和维持了庞大的全球科学和工程用户基础。

Science and engineering research increasingly relies on repeated execution of a complex series of steps (i.e., workflows) to form hypotheses; conduct experiments; analyze results; and refine theory. Computation is often essential throughout the workflow and in this case, software can improve productivity by managing the computational and data workflow. Swift is one such open-source workflow system that has been developed and widely used in diverse areas ranging from materials simulations and climate modeling to neuroscience and genomics. This project extends the capabilities of Swift by integrating it with other software systems that enable collaboration, usability, maintainability, and productivity. The new ecosystem, Swift/E, will enable scientists and engineers to more productively create and run computational workflow campaigns of larger scale, and debug, execute, adapt, and disseminate them faster and easier than has been possible to date. These workflows embody and communicate the computational methods specific to each domain of scientific inquiry. Swift/E achieves community engagement and extensive productivity benefits for a large user community through an integrated program of research, education, and software dissemination. The project engages and serves science and engineering communities by creating patterns of practice for building and sharing reusable workflow libraries, and by training students, educators, and researchers in their use. To advance the education of the next generation of computationally trained scientists, Swift/E powers a network of NSF-supported "e-Labs" that teach the concepts of collaborative parallel computational science at high school and undergraduate levels, reaching over a thousand students annually.The open-source Swift/E "ecosystem" integrates Swift with several scientific software elements that play a major role in the national and global cyberinfrastructure of today. These elements are: Swift for the parallel scripting of scientific workflow; Globus for data cataloging, management, and high-speed wide-area transport; the Web-based Galaxy workflow portal for workflow composition, execution, and collaborative sharing; Jupyter for the interactive development, testing, debugging, and assembly of high level programming and workflow languages; Python and R for productively expressing high-level computational logic; and "git" and related tools and Web portals for revision control, code dissemination and sharing, and for the collaborative engagement of developers. Swift's implicitly parallel programming language is minimal and compact. Swift provides a facility for embedding other scripting languages (currently Python, R, Julia and Tcl) into its runtime environment. This project merges newer extreme-scale "Swift/T" capabilities with the flexible and portable original "Swift/K" version to make the core Swift/E software element more powerful and flexible while lowering it?s ongoing support cost. Swift/E enhances usability by extending Swift's troubleshooting and inter-language integration facilities. And with enhanced and innovative workflow sharing archives, new training materials, and a sustained program for user support and self-sustaining and expanding community engagement, the Swift/E project engages, supports, and sustains a large global science and engineering user base.