Understanding and improving the quality and reproducibility of Jupyter notebooks.
Understanding and improving the quality and reproducibility of Jupyter notebooks.
复制标题
DOI：
10.1007/s10664-021-09961-9
复制
发表时间：
2021
影响因子：
4.1
通讯作者：
Freire J
中科院分区：
文献类型：
--
作者：
Pimentel JF;Murta L;Braganholo V;Freire J
在科学界和工业界，许多不同的社区都广泛采用了XYTER笔记本电脑。它们支持创建将联合收割机代码、文本和执行结果与可视化和其他丰富媒体结合在一起的文字编程文档。自记录方面和再现结果的能力被吹捧为笔记本的显著优点。与此同时，越来越多的批评认为，笔记本的使用方式导致了意想不到的行为，鼓励了糟糕的编码实践，并使其结果难以重现。为了更好地理解真实的笔记本开发中使用的好的和坏的实践，在之前的工作中，我们研究了来自GitHub的140万个笔记本。我们对影响再现性的特征进行了详细分析，提出了可以提高再现性的最佳实践，并讨论了需要进一步研究和开发的开放性挑战。在本文中，我们以四种不同的方式扩展了分析，以验证我们最初研究中发现的假设。首先，我们将一组受欢迎的笔记本电脑分开，以检查是否得到更多关注的笔记本电脑具有更高的质量和可复制性。其次，我们从完整的数据集中抽取笔记本，对数据集的组成以及它们具有哪些功能进行深入的定性分析。第三，我们通过隔离库依赖和测试不同的执行顺序进行了更详细的分析。我们报告这些因素如何影响再现率。最后，我们从笔记本中挖掘关联规则。我们讨论我们发现的模式，这提供了笔记本电脑的可重复性的额外见解。根据我们的发现和我们提出的最佳实践，我们设计了Julynter，这是一个可识别笔记本电脑中潜在问题并提出改进建议以提高其重现性的实验室扩展。我们通过远程用户实验来评估Julynter，目的是评估Julynter的建议和可用性。
Jupyter Notebooks have been widely adopted by many different communities, both in science and industry. They support the creation of literate programming documents that combine code, text, and execution results with visualizations and other rich media. The self-documenting aspects and the ability to reproduce results have been touted as significant benefits of notebooks. At the same time, there has been growing criticism that the way in which notebooks are being used leads to unexpected behavior, encourages poor coding practices, and makes it hard to reproduce its results. To better understand good and bad practices used in the development of real notebooks, in prior work we studied 1.4 million notebooks from GitHub. We presented a detailed analysis of their characteristics that impact reproducibility, proposed best practices that can improve the reproducibility, and discussed open challenges that require further research and development. In this paper, we extended the analysis in four different ways to validate the hypothesis uncovered in our original study. First, we separated a group of popular notebooks to check whether notebooks that get more attention have more quality and reproducibility capabilities. Second, we sampled notebooks from the full dataset for an in-depth qualitative analysis of what constitutes the dataset and which features they have. Third, we conducted a more detailed analysis by isolating library dependencies and testing different execution orders. We report how these factors impact the reproducibility rates. Finally, we mined association rules from the notebooks. We discuss patterns we discovered, which provide additional insights into notebook reproducibility. Based on our findings and best practices we proposed, we designed Julynter, a Jupyter Lab extension that identifies potential issues in notebooks and suggests modifications that improve their reproducibility. We evaluate Julynter with a remote user experiment with the goal of assessing Julynter recommendations and usability.
登录
查看更多内容
影响因子：
4.1
作者：
Arnaoudova, Venera;Di Penta, Massimiliano;Antoniol, Giuliano
通讯作者：
Antoniol, Giuliano
影响因子：
3.5
作者：
Garousi, Vahid;Kucuk, Baris
通讯作者：
Kucuk, Baris
影响因子：
1.4
作者：
KNUTH, DE
通讯作者：
KNUTH, DE
影响因子：
2.1
作者：
Freire, Juliana;Koop, David;Silva, Claudio T.
通讯作者：
Silva, Claudio T.
DOI：
10.1080/10447310802205776
发表时间：
2008-08-01
影响因子：
4.7
作者：
Bangor, Aaron;Kortum, Philip T.;Miller, James T.
通讯作者：
Miller, James T.