Dagger: A Data (not code) Debugger
Dagger: A Data (not code) Debugger
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
E. Rezig;Lei Cao;Giovanni Simonini;Maxime Schoemans;S. Madden;N. Tang;M. Ouzzani;M. Stonebraker
中科院分区：
文献类型：
--
作者：
E. Rezig;Lei Cao;Giovanni Simonini;Maxime Schoemans;S. Madden;N. Tang;M. Ouzzani;M. Stonebraker
随着数据科学库和框架的民主化，大多数数据科学家使用脚本集合（例如，Python，R）。这标志着传统应用程序与存储和管理应用程序数据的DBMS来回通信的转变。虽然代码调试器在过去几十年中已经达到了令人印象深刻的成熟度，但它们在帮助用户探索数据驱动的假设场景（例如，将训练集分成两部分并构建两个ML模型）。这些场景虽然在编程上可行，但对于用户管理自己来说是一个巨大的负担。Dagger（Data Debugger）是一个端到端的数据调试器，它抽象了以数据为中心的关键原语，使用户能够快速识别和缓解给定管道中与数据相关的问题。Dagger的动机是我们与几个组织的数据科学家进行的一系列访谈。Dagger的一个初步版本已经被整合到Data Civilizer 2.0中，以帮助马萨诸塞州总医院的医生处理复杂的管道。
With the democratization of data science libraries and frameworks, most data scientists manage and generate their data analytics pipelines using a collection of scripts (e.g., Python, R). This marks a shift from traditional applications that communicate back and forth with a DBMS that stores and manages the application data. While code debuggers have reached impressive maturity over the past decades, they fall short in assisting users to explore data-driven what-if scenarios (e.g., split the training set into two and build two ML models). Those scenarios, while doable programmati-cally, are a substantial burden for users to manage themselves. Dagger (Data Debugger) is an end-to-end data de-bugger that abstracts key data-centric primitives to enable users to quickly identify and mitigate data-related problems in a given pipeline. Dagger was motivated by a series of interviews we conducted with data scientists across several organizations. A preliminary version of Dagger has been incorporated into Data Civilizer 2.0 to help physicians at the Massachusetts General Hospital process complex pipelines.