Big Data Provenance: Challenges, State of the Art and Opportunities.
Big Data Provenance: Challenges, State of the Art and Opportunities.
复制标题
DOI：
10.1109/bigdata.2015.7364047
复制
发表时间：
2015-10
期刊：
影响因子：
--
通讯作者：
Altintas I
中科院分区：
文献类型：
--
作者：
Wang J;Crawl D;Purawat S;Nguyen M;Altintas I
追踪出处的能力是支持数据谱系和再现性的科学工作流程的一个关键特征。大数据的数量、种类和速度带来的挑战，也对大数据的来源和质量(定义为准确性)构成了相关挑战。分布式大数据来源信息的规模和种类不断增加，在整个来源生命周期中带来了新的技术挑战和机遇，包括记录、查询、共享和利用。本文讨论了与数据集本身的准确性以及分析这些数据集的分析流程的来源相关的大数据来源方面的挑战和机遇。它还解释了我们目前在跟踪和利用大数据来源方面所做的努力，使用工作流作为分析大数据的编程模型。
Ability to track provenance is a key feature of scientific workflows to support data lineage and reproducibility. The challenges that are introduced by the volume, variety and velocity of Big Data, also pose related challenges for provenance and quality of Big Data, defined as veracity. The increasing size and variety of distributed Big Data provenance information bring new technical challenges and opportunities throughout the provenance lifecycle including recording, querying, sharing and utilization. This paper discusses the challenges and opportunities of Big Data provenance related to the veracity of the datasets themselves and the provenance of the analytical processes that analyze these datasets. It also explains our current efforts towards tracking and utilizing Big Data provenance using workflows as a programming model to analyze Big Data.