LAMP: data provenance for graph based machine learning algorithms through derivative computation
LAMP: data provenance for graph based machine learning algorithms through derivative computation
复制标题
DOI：
10.1145/3106237.3106291
复制
发表时间：
2017-08
期刊：
影响因子：
--
通讯作者：
Shiqing Ma;Yousra Aafer;Zhaogui Xu;Wen-Chuan Lee;Juan Zhai;Yingqi Liu;X. Zhang
中科院分区：
文献类型：
--
作者：
Shiqing Ma;Yousra Aafer;Zhaogui Xu;Wen-Chuan Lee;Juan Zhai;Yingqi Liu;X. Zhang
数据出处跟踪确定与给定输出相关的输入集。它可以在数据工程中进行质量控制和问题诊断。大多数现有技术通过跟踪程序依赖性来工作。他们无法定量评估相关输入的重要性，这对于机器学习算法至关重要，在机器学习算法中，其中的输出倾向于取决于一系列大量输入，而其中一些非常重要。在本文中，我们提出了一种用于机器学习算法的出处计算系统。受自动分化（AD）的启发，LAMP通过计算部分导数量化输入对于输出的重要性。 LAMP将原始数据处理和更昂贵的导数计算分开，以实现成本效益。此外，它允许量化与离散行为有关的输入（例如控制流选择）的重要性。对一组现实世界计划和数据集的评估表明，与基于程序依赖的技术相比，灯产生更精确和简洁的出处，开销要少得多。我们的案例研究表明，灯在数据工程中的问题诊断中的潜力。
Data provenance tracking determines the set of inputs related to a given output. It enables quality control and problem diagnosis in data engineering. Most existing techniques work by tracking program dependencies. They cannot quantitatively assess the importance of related inputs, which is critical to machine learning algorithms, in which an output tends to depend on a huge set of inputs while only some of them are of importance. In this paper, we propose LAMP, a provenance computation system for machine learning algorithms. Inspired by automatic differentiation (AD), LAMP quantifies the importance of an input for an output by computing the partial derivative. LAMP separates the original data processing and the more expensive derivative computation to different processes to achieve cost-effectiveness. In addition, it allows quantifying importance for inputs related to discrete behavior, such as control flow selection. The evaluation on a set of real world programs and data sets illustrates that LAMP produces more precise and succinct provenance than program dependence based techniques, with much less overhead. Our case studies demonstrate the potential of LAMP in problem diagnosis in data engineering.