On the Real-World Effectiveness of Static Bug Detectors at Finding Null Pointer Exceptions
On the Real-World Effectiveness of Static Bug Detectors at Finding Null Pointer Exceptions
复制标题
DOI：
10.1109/ase51524.2021.9678535
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
David A. Tomassi;Cindy Rubio-González
中科院分区：
文献类型：
--
作者：
David A. Tomassi;Cindy Rubio-González
静态bug检测器旨在帮助开发人员自动发现和预防bug。在这篇经验论文中，我们研究了静态错误检测器在识别错误指针引用或错误指针引用（NPE）方面的有效性。NPE遍布从系统到Web开发的所有编程领域。具体来说，我们的研究测量了五个Java静态错误检测器的有效性：BikkerFramework，ERADICATE，INFER，NULLAWAY和SPOTBUGS。我们对来自BUGSWARM和DEFECTS4J数据集中的42个开源项目的102个真实世界和可重复的NPE进行了研究。我们应用两个已知的方法来确定是否一个错误被发现的给定的工具，并引入两个新的方法，利用堆栈跟踪和代码覆盖率信息。此外，我们还提供了工具功能和错误特征的分类，以更好地理解工具的优点和缺点。总体而言，研究中的工具只发现了102个错误中的30个（29.4%），其中大多数是由ERADICATE发现的。根据我们的观察，我们确定并讨论了使这些工具更加有效和有用的机会。
Static bug detectors aim at helping developers to automatically find and prevent bugs. In this experience paper, we study the effectiveness of static bug detectors at identifying Null Pointer Dereferences or Null Pointer Exceptions (NPEs). NPEs pervade all programming domains from systems to web development. Specifically, our study measures the effectiveness of five Java static bug detectors: CheckerFramework, ERADICATE, INFER, NULLAWAY, and SPOTBUGS. We conduct our study on 102 real-world and reproducible NPEs from 42 open-source projects found in the BUGSWARM and DEFECTS4J datasets. We apply two known methods to determine whether a bug is found by a given tool, and introduce two new methods that leverage stack trace and code coverage information. Additionally, we provide a categorization of the tool’s capabilities and the bug characteristics to better understand the strengths and weaknesses of the tools. Overall, the tools under study only find 30 out of 102 bugs (29.4%), with the majority found by ERADICATE. Based on our observations, we identify and discuss opportunities to make the tools more effective and useful.