Provenance for seismological processing pipelines in a distributed streaming workflow
Provenance for seismological processing pipelines in a distributed streaming workflow
复制标题
分布式流工作流程中地震处理管道的来源
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
M. Atkinson
中科院分区：
文献类型：
--
作者：
A. Spinuso;J. Cheney;M. Atkinson
获取流工作流的来源带来了与高更新率和执行的大分布相关的挑战，这可以分布在几个机构基础结构中。此外，每个转换步骤产生的典型的大量数据不能总是有效地存储和保存。这可能对结果的评估（例如实时评估）构成障碍，这表明了可定制元数据提取过程的重要性。在本文中，我们提出了在地震学领域的用例驱动场景中解决上述来源挑战的方法，该场景需要在大型数据流上执行处理管道。特别地，我们将讨论工作流中可编程方法的当前实现和即将面临的挑战，以实现来源跟踪，构建复合功能，选择性记录和特定领域的元数据生成。
Harvesting provenance for streaming workflows presents challenges related to the high rate of the updates and a large distribution of the execution, which can be spread across several institutional infrastructures. Moreover, the typically large volume of data produced by each transformation step can not be always stored and preserved efficiently. This can represent an obstacle for the evaluation of the results, for instance, in real-time, suggesting the importance of customisable metadata extraction procedures. In this paper we present our approach to the aforementioned provenance challenges within a use-case driven scenario in the field of seismology, which requires the execution of processing pipelines over a large datastream. In particular, we will discuss the current implementation and the upcoming challenges for an in-worfklow programmatic approach to provenance tracing, building on composite functions, selective recording and domain specific metadata production.
用于科学知识发现的数据密集型架构
影响因子：
1.2
作者：
Atkinson M
通讯作者：
Atkinson M