The Workflow Trace Archive: Open-Access Data From Public and Private Computing Infrastructures
The Workflow Trace Archive: Open-Access Data From Public and Private Computing Infrastructures
复制标题
DOI：
10.1109/tpds.2020.2984821
复制
发表时间：
2019-06
影响因子：
5.3
通讯作者：
L. Versluis;Roland Mathá;Sacheendra Talluri;T. Hegeman;R. Prodan;E. Deelman;A. Iosup
中科院分区：
文献类型：
--
作者：
L. Versluis;Roland Mathá;Sacheendra Talluri;T. Hegeman;R. Prodan;E. Deelman;A. Iosup
现实的、相关的和可重复的实验通常需要从真实世界环境中收集的输入轨迹。在这项工作中，我们专注于工作流的跟踪-常见于数据中心、云和HPC基础架构。我们表明，国家的最先进的使用工作流的痕迹提出了重要的问题：（1）使用现实的痕迹是罕见的，（2）使用现实的，开放访问的痕迹，甚至更多。为了缓解这些问题，我们引入了工作流跟踪存档（WTA），这是一个开放访问的存档，可以从不同的计算基础设施和工具中解析、验证和分析跟踪。WTA包括从${>}10$>10计算基础设施捕获的${>}48$>48百万个工作流，代表了广泛多样的跟踪域和特征。为了强调痕量多样性的重要性，我们表征了WTA含量，并在模拟中分析了痕量多样性对实验结果的影响。我们的研究结果表明，工作负载源，域和字段之间的特征，属性和工作流结构的显着差异。
Realistic, relevant, and reproducible experiments often need input traces collected from real-world environments. In this work, we focus on traces of workflows—common in datacenters, clouds, and HPC infrastructures. We show that the state-of-the-art in using workflow-traces raises important issues: (1) the use of realistic traces is infrequent and (2) the use of realistic, open-access traces even more so. Alleviating these issues, we introduce the Workflow Trace Archive (WTA), an open-access archive of workflow traces from diverse computing infrastructures and tooling to parse, validate, and analyze traces. The WTA includes ${>}48$>48 million workflows captured from ${>}10$>10 computing infrastructures, representing a broad diversity of trace domains and characteristics. To emphasize the importance of trace diversity, we characterize the WTA contents and analyze in simulation the impact of trace diversity on experiment results. Our results indicate significant differences in characteristics, properties, and workflow structures between workload sources, domains, and fields.