An empirical study of the robustness of MacOS applications using random testing
An empirical study of the robustness of MacOS applications using random testing
复制标题
DOI：
10.1145/1228291.1228308
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
B. Miller;Gregory Cooksey;Fredrick Moore
中科院分区：
文献类型：
--
作者：
B. Miller;Gregory Cooksey;Fredrick Moore
我们报告的第四个在一系列的研究应用程序的可靠性，面对随机输入。在过去的15年里，我们研究了UNIX命令行和基于X-Window（GUI）的应用程序以及Windows应用程序的可靠性。在这项研究中，我们将我们的模糊测试技术应用于Mac OS X操作系统上运行的应用程序。我们继续使用一种简单的，甚至是过于简单的技术：非结构化的黑盒随机测试，将失败视为崩溃或挂起。在前面的三个研究中，该技术是粗糙的，但似乎是有效的定位错误在真实的programmes.We测试了135命令行UNIX实用程序和Mac OS X上的30个图形应用程序的可靠性，通过馈送随机输入到每个。我们报告应用程序的故障-崩溃（转储核心）或挂起（无限期循环）-并且，在源代码可用的情况下，我们确定这些故障的原因并对其进行分类。我们的测试仅使7%的命令行实用程序崩溃，这比以前研究中几乎所有情况下观察到的故障率要低得多。我们发现基于GUI的应用程序不太可靠：在我们测试的30个应用程序中，只有8个没有崩溃或挂起。另外20人坠毁，2人挂起。这些GUI结果明显差于之前的Windows（Win32）或UNIX（X-Windows）研究。
We report on the fourth in a series of studies on the reliability of application programs in the face of random input. Over the previous 15 years, we have studied the reliability of UNIX command line and X-Window based (GUI) applications and Windows applications. In this study, we apply our fuzz testing techniques to applications running on the Mac OS X operating system. We continue to use a simple, or even simplistic technique: unstructured black-box random testing, considering a failure to be a crash or hang. As in the previous three studies, the technique is crude but seems to be effective in locating bugs in real programs.We tested the reliability of 135 command-line UNIX utilities and thirty graphical applications on Mac OS X by feeding random input to each. We report on application failures - crashes (dumps core) or hangs (loops indefinitely) - and, where source code is available, we identify the causes of these failures and categorize them.Our testing crashed only 7% of the command-line utilities, a considerably lower rate of failure than observed in almost all cases of previous studies. We found the GUI-based applications to be less reliable: of the thirty that we tested, only eight did not crash or hang. Twenty others crashed, and two hung. These GUI results were noticeably worse than either of the previous Windows (Win32) or UNIX (X-Windows) studies.