Data distribution debugging in machine learning pipelines
Data distribution debugging in machine learning pipelines
复制标题
DOI：
10.1007/s00778-021-00726-w
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Stefan Grafberger;Paul Groth;Julia Stoyanovich;Sebastian Schelter
中科院分区：
文献类型：
--
作者：
Stefan Grafberger;Paul Groth;Julia Stoyanovich;Sebastian Schelter
机器学习（ML）越来越多地用于自动化有影响力的决策，这种广泛使用所带来的风险正在引起政策制定者、科学家和媒体的关注。ML应用程序的输入数据往往很脆弱，这导致了对它们的正确性、可靠性和公平性的担忧。在本文中，我们描述了emlinspect，这是一个帮助诊断和减轻ML管道中预处理步骤中可能出现的技术偏差的库。我们把这些问题统称为数据分发错误。其关键思想是从预处理管道中提取一个有向无环图表示的卷积，并使用此表示自动仪器的代码与预定义的检查。这些检查是基于一个轻量级的注释传播方法，以传播元数据，如血统信息从操作员到操作员。与现有的工作相比，mlinspectoporoperates流行的数据科学库的声明性抽象，如估计器/Transformer管道，不需要手动代码插装。我们讨论的设计和实现themlinks库，并给出了一个全面的端到端的例子，说明其功能。
Machine learning (ML) is increasingly used to automate impactful decisions, and the risks arising from this widespread use are garnering attention from policy makers, scientists, and the media. ML applications are often brittle with respect to their input data, which leads to concerns about their correctness, reliability, and fairness. In this paper, we describemlinspect, a library that helps diagnose and mitigate technical bias that may arise during preprocessing steps in an ML pipeline. We refer to these problems collectively asdata distribution bugs. The key idea is to extract a directed acyclic graph representation of the dataflow from a preprocessing pipeline and to use this representation to automatically instrument the code with predefinedinspections. These inspections are based on a lightweight annotation propagation approach to propagate metadata such as lineage information from operator to operator. In contrast to existing work,mlinspectoperates on declarative abstractions of popular data science libraries like estimator/transformer pipelines and does not require manual code instrumentation. We discuss the design and implementation of themlinspectlibrary and give a comprehensive end-to-end example that illustrates its functionality.