Workflows Community Summit 2022: A Roadmap Revolution
Workflows Community Summit 2022: A Roadmap Revolution
复制标题
2022 年工作流程社区峰会：路线图革命
DOI：
10.5281/zenodo.7750670
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
M. Zulfiqar
中科院分区：
文献类型：
--
作者：
Rafael Ferreira da Silva;Rosa M. Badia;Venkat Bala;Deborah Bard;P. Bremer;Ian Buckley;Silvina Caíno;K. Chard;C. Goble;S. Jha;D. Katz;D. Laney;M. Parashar;F. Suter;N. Tyler;T. Uram;I. Altintas;S. Andersson;W. Arndt;J. Aznar;Jonathan Bader;B. Baliś;Chris E. Blanton;K. Braghetto;Aharon Brodutch;Paul Brunk;H. Casanova;Alba Cervera Lierta;Justin Chigu;T. Coleman;Nick Collier;Iacopo Colonnelli;Frederik Coppens;M. Crusoe;W. Cunningham;Bruno Kinoshita;Paolo Di Tommaso;C. Doutriaux;M. Downton;W. Elwasif;B. Enders;Chris Erdmann;T. Fahringer;Ludmilla Figueiredo;Rosa Filgueira;M. Foltin;A. Fouilloux;Luiz M. R. Gadelha;Andrew Gallo;A. G. Saez;D. Garijo;R. Gerlach;Ryan E. Grant;Samuel Grayson;Patricia A. Grubel;Johan O. R. Gustafsson;Valérie Hayot;Oscar R. Hernandez;Marcus Hilbrich;Annmary Justine;I. Laflotte;Fabian Lehmann;André Luckow;Jakob Luettgau;K. Maheshwari;Motohiko Matsuda;Doriana Medic;P. Mendygral;M. Michalewicz;J. Nonaka;Maciej Pawlik;L. Pottier;Line C. Pouchard;Mathias Putz;Santosh Kumar Radha;L. Ramakrishnan;S. Ristov;P. Romano;Daniel Rosendo;M. Ruefenacht;Katarzyna Rycerz;Nishant Saurabh;V. Savchenko;Martin Schulz;C. Simpson;R. Sirvent;Tyler J. Skluzacek;S. Soiland;Renan Souza;S. Sukumar;Ziheng Sun;A. Sussman;D. Thain;Mikhail Titov;Benjamín Tovar;Aalap Tripathy;M. Turilli;Bartosz Tuznik;H. V. Dam;Aurelio Vivas;Logan T. Ward;Patrick M. Widener;Sean R. Wilkinson;Justyna Zawalska;M. Zulfiqar
科学工作流已成为广泛的科学计算用例中不可或缺的工具。科学发现越来越依赖于工作流程来协调大型复杂的科学实验，这些实验的范围从基于云的数据预处理管道的执行到多设施仪器到边缘到HPC的计算工作流程。考虑到科学计算的不断变化以及新兴科学应用不断变化的需求，开发新的科学工作流程和系统功能以提高现有系统和应用的效率、弹性和普及性至关重要。具体来说，机器学习/人工智能（ML/AI）工作流的激增，对处理边缘仪器产生的大规模数据集的需求，近实时数据处理的强化，对长期实验活动的支持，以及量子计算作为HPC的辅助手段的出现，都显著改变了工作流系统的功能和操作要求。例如，工作流系统现在需要支持从边缘到云到HPC的数据流，以便管理许多小型文件，在确保高准确性的同时允许数据减少，协调分布式服务（工作流，仪器，数据移动，出处，发布等）。跨计算和用户设施等。此外，为了加速科学发展，这些系统还必须实现规范/标准和API，以实现系统和应用程序之间的无缝（水平和垂直）集成，并根据FAIR原则发布工作流程及其相关产品。本文件报告了2022年11月29日和30日举行的2022年国际版工作流社区峰会的讨论和结果。
Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex scientific experiments that range from execution of a cloud-based data preprocessing pipeline to multi-facility instrument-to-edge-to-HPC computational workflows. Given the changing landscape of scientific computing and the evolving needs of emerging scientific applications, it is paramount that the development of novel scientific workflows and system functionalities seek to increase the efficiency, resilience, and pervasiveness of existing systems and applications. Specifically, the proliferation of machine learning/artificial intelligence (ML/AI) workflows, need for processing large scale datasets produced by instruments at the edge, intensification of near real-time data processing, support for long-term experiment campaigns, and emergence of quantum computing as an adjunct to HPC, have significantly changed the functional and operational requirements of workflow systems. Workflow systems now need to, for example, support data streams from the edge-to-cloud-to-HPC enable the management of many small-sized files, allow data reduction while ensuring high accuracy, orchestrate distributed services (workflows, instruments, data movement, provenance, publication, etc.) across computing and user facilities, among others. Further, to accelerate science, it is also necessary that these systems implement specifications/standards and APIs for seamless (horizontal and vertical) integration between systems and applications, as well as enabling the publication of workflows and their associated products according to the FAIR principles. This document reports on discussions and findings from the 2022 international edition of the Workflows Community Summit that took place on November 29 and 30, 2022.
登录
查看更多内容
面向在遥感中使用异构系统的动态可组合性方法
DOI：
10.1109/escience55777.2022.00047
发表时间：
2022
期刊：
2022 IEEE 18th International Conference on e-Science (e-Science
影响因子：
--
作者：
Altintas, Ilkay;Perez, Ismael;Mishin, Dmitry;Trouillaud, Adrien;Irving, Christopher;Graham, John;Tatineni, Mahidhar;DeFanti, Thomas;Strande, Shawn;Smarr, Larry
通讯作者：
Smarr, Larry
DOI：
10.14778/2732977.2732988
发表时间：
2014-08
期刊：
Proc. VLDB Endow.
影响因子：
--
作者：
J. Starlinger;Bryan Brancotte;Sarah Cohen Boulakia;U. Leser
通讯作者：
J. Starlinger;Bryan Brancotte;Sarah Cohen Boulakia;U. Leser
用于分布式批量元数据提取的无服务器框架
DOI：
10.1145/3431379.3460636
发表时间：
2021
期刊：
Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing (HPDC
影响因子：
--
作者：
Skluzacek, Tyler J.;Wong, Ryan;Li, Zhuozhao;Chard, Ryan;Chard, Kyle;Foster, Ian
通讯作者：
Foster, Ian
DOI：
10.1177/1094342020935991
发表时间：
2020
期刊：
The International Journal of High Performance Computing Applications
影响因子：
--
作者：
Childs, Hank;Ahern, Sean D.;Ahrens, James;Bauer, Andrew C.;Bennett, Janine;Bethel, E. Wes;Bremer, Peer-Timo;Brugger, Eric;Cottam, Joseph;Dorier, Matthieu
通讯作者：
Dorier, Matthieu
WfChef：自动生成准确的科学工作流程生成器
DOI：
--
发表时间：
2021
期刊：
17th IEEE EScience Conference
影响因子：
--
作者：
Coleman, Taina;Casanova, Henri;Ferreira da Silva, Rafael
通讯作者：
Ferreira da Silva, Rafael