Fixing dependency errors for Python build reproducibility
Fixing dependency errors for Python build reproducibility
复制标题
DOI：
10.1145/3460319.3464797
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Suchita Mukherjee;Abigail Almanza;Cindy Rubio-González
中科院分区：
文献类型：
--
作者：
Suchita Mukherjee;Abigail Almanza;Cindy Rubio-González
软件可重复性对于可重复使用性和研究累积进展很重要。不可生育软件的重要表现是软件随时间构建的结果发生了变化。在增强代码重复使用的同时，使用在PYPI等集中存储库上托管的开源依赖软件包可能会对构建可重复性产生不利影响。这些软件包的频繁更新通常会导致其最新版本的应用程序进行破坏的更改。由于Python依赖性的广泛使用以及缺乏依赖性版本规范的统一实践，因此大型Python应用程序可能会导致其历史构建变得无法复制。手动固定依赖性错误需要昂贵的开发人员时间和精力，而自动化方法则面临解析非结构化构建日志，查找及其依赖性以及探索依赖性版本的指数搜索空间的挑战。在本文中，我们调查了开源Python项目如何指定依赖性版本，以及它们的可重复性如何受到依赖包的影响。我们提出了一个工具pydfix，以检测和修复由依赖性错误引起的python构建中的不可复制性。 Pydfix在两个错误数据集中评估了Bugswarm和Bugsinpy，这两者都是由现实世界中的开源项目构建的。 Pydfix总共分析了2,702个构建，由于依赖性误差，确定其中1,921（71.1％）是无法可复制的。通过这些，Pydfix为859（44.7％）的构建提供了完整的修复程序，并为另外632（32.9％）的构建提供了部分修复。
Software reproducibility is important for re-usability and the cumulative progress of research. An important manifestation of unreproducible software is the changed outcome of software builds over time. While enhancing code reuse, the use of open-source dependency packages hosted on centralized repositories such as PyPI can have adverse effects on build reproducibility. Frequent updates to these packages often cause their latest versions to have breaking changes for applications using them. Large Python applications risk their historical builds becoming unreproducible due to the widespread usage of Python dependencies, and the lack of uniform practices for dependency version specification. Manually fixing dependency errors requires expensive developer time and effort, while automated approaches face challenges of parsing unstructured build logs, finding transitive dependencies, and exploring an exponential search space of dependency versions. In this paper, we investigate how open-source Python projects specify dependency versions, and how their reproducibility is impacted by dependency packages. We propose a tool PyDFix to detect and fix unreproducibility in Python builds caused by dependency errors. PyDFix is evaluated on two bug datasets BugSwarm and BugsInPy, both of which are built from real-world open-source projects. PyDFix analyzes a total of 2,702 builds, identifying 1,921 (71.1%) of them to be unreproducible due to dependency errors. From these, PyDFix provides a complete fix for 859 (44.7%) builds, and partial fixes for an additional 632 (32.9%) builds.