Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset
Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset
复制标题
DOI：
10.1007/s10664-016-9470-4
复制
发表时间：
2016-10
影响因子：
4.1
通讯作者：
Matias Martinez;Thomas Durieux;Romain Sommerard;J. Xuan;Monperrus Martin
中科院分区：
文献类型：
--
作者：
Matias Martinez;Thomas Durieux;Romain Sommerard;J. Xuan;Monperrus Martin
Defects4J是一个大型的、同行评审的、结构化的Java bug数据集。Defects4J中的每个bug都有一个测试套件和至少一个触发bug的失败测试用例。在本文中，我们报告了一个实验，以探讨基于自动测试套件的缺陷4J修复的有效性。我们的实验结果表明，所考虑的国家的最先进的修复方法可以生成补丁的224个错误中的47。然而，这些补丁仅适用于测试套件，这意味着它们通过了测试套件，并且可能不正确，超出了测试套件满意正确性标准。我们手动分析了84个不同的补丁，以评估它们的真实的正确性。总的来说，9个真实的Java bug可以通过基于测试套件的修复来正确修复。该分析表明，基于测试套件的修复遭受未指定的错误，其中微不足道或不正确的补丁仍然通过测试套件。在实用性方面，找到一个补丁平均需要14.8分钟。实验是在科学网格上进行的，总共计算时间为17.6天。所有的修复系统和实验结果都在Github上公开，以方便将来对自动修复的研究。
Defects4J is a large, peer-reviewed, structured dataset of real-world Java bugs. Each bug in Defects4J comes with a test suite and at least one failing test case that triggers the bug. In this paper, we report on an experiment to explore the effectiveness of automatic test-suite based repair on Defects4J. The result of our experiment shows that the considered state-of-the-art repair methods can generate patches for 47 out of 224 bugs. However, those patches are only test-suite adequate, which means that they pass the test suite and may potentially be incorrect beyond the test-suite satisfaction correctness criterion. We have manually analyzed 84 different patches to assess their real correctness. In total, 9 real Java bugs can be correctly repaired with test-suite based repair. This analysis shows that test-suite based repair suffers from under-specified bugs, for which trivial or incorrect patches still pass the test suite. With respect to practical applicability, it takes on average 14.8 minutes to find a patch. The experiment was done on a scientific grid, totaling 17.6 days of computation time. All the repair systems and experimental results are publicly available on Github in order to facilitate future research on automatic repair.