Subtle Bugs Everywhere: Generating Documentation for Data Wrangling Code
Subtle Bugs Everywhere: Generating Documentation for Data Wrangling Code
复制标题
DOI：
10.1109/ase51524.2021.9678520
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Chenyang Yang;Shurui Zhou;Jin L. C. Guo;Christian Kästner
中科院分区：
文献类型：
--
作者：
Chenyang Yang;Shurui Zhou;Jin L. C. Guo;Christian Kästner
据报道，数据科学家在日常工作中花费大量时间进行数据争论，即清理数据和提取特征。然而，数据争用代码通常是重复的，并且编写时容易出错。此外，在重用和采用现有代码时很容易引入细微的错误，这会导致模型质量降低。为了支持数据科学家进行数据争论，我们提出了一种为数据争论代码生成文档的技术。我们使用（1）程序合成技术来自动总结数据转换和（2）测试用例选择技术，有目的地选择有代表性的例子，从数据的执行信息的基础上收集定制的动态程序分析。我们证明了，一个mixyterLab扩展与我们的技术可以提供按需文档的许多细胞在流行的笔记本电脑，并发现在用户研究中，用户与我们的插件更快，更有效地找到现实的数据争吵代码中的错误。
Data scientists reportedly spend a significant amount of their time in their daily routines on data wrangling, i.e. cleaning data and extracting features. However, data wrangling code is often repetitive and error-prone to write. Moreover, it is easy to introduce subtle bugs when reusing and adopting existing code, which results in reduced model quality. To support data scientists with data wrangling, we present a technique to generate documentation for data wrangling code. We use (1) program synthesis techniques to automatically summarize data transformations and (2) test case selection techniques to purposefully select representative examples from the data based on execution information collected with tailored dynamic program analysis. We demonstrate that a JupyterLab extension with our technique can provide on-demand documentation for many cells in popular notebooks and find in a user study that users with our plugin are faster and more effective at finding realistic bugs in data wrangling code.