Rise of the HaCRS: Augmenting Autonomous Cyber Reasoning Systems with Human Assistance
Rise of the HaCRS: Augmenting Autonomous Cyber Reasoning Systems with Human Assistance
复制标题
DOI：
10.1145/3133956.3134105
复制
发表时间：
2017-08
期刊：
影响因子：
--
通讯作者：
Yan Shoshitaishvili;Michael Weissbacher;Lukas Dresel;Christopher Salls;Ruoyu Wang;Christopher Krügel;Giovanni Vigna
中科院分区：
文献类型：
--
作者：
Yan Shoshitaishvili;Michael Weissbacher;Lukas Dresel;Christopher Salls;Ruoyu Wang;Christopher Krügel;Giovanni Vigna
软件渗透到我们世界的方方面面，从我们的家到提供关键任务服务的基础设施。随着软件系统的规模和复杂性的增加，软件安全缺陷的数量和复杂性也在增加。对这些缺陷的分析开始是一种手工方法，但很快就发现，单靠手工方法是无法扩展的，并且需要工具来协助人类专家完成这项任务，从而产生了许多技术和方法，这些技术和方法自动化了漏洞分析过程的某些方面。最近，DARPA开展了网络大挑战，这是一项自主漏洞分析系统之间的竞赛，旨在将工具辅助的以人为中心的范式推向完全自动化的领域，希望通过消除人为因素，分析能够达到新的高度。然而，当自主系统与人类专家进行竞争时，很明显，某些任务虽然简单，但无法由自主系统执行，因为它们需要理解所分析应用程序的逻辑。基于这一观察，我们提出了脆弱性分析范式的转变，从工具辅助的以人为中心向以人为辅助的以工具为中心转变。在此范例中，自动化系统编排漏洞分析过程，并利用人类（具有不同级别的专业知识）来执行定义良好的子任务，其结果集成到分析中。因此，可以将分析扩展到更多的程序，同时优化昂贵的人力资源的使用。在本文中，我们详细介绍了人类辅助自动化漏洞分析系统的设计，描述了其在参与网络大挑战的开源自主漏洞分析系统上的实现，并评估和讨论了非专家人类辅助可以为自动化分析方法提供的重大改进。
Software permeates every aspect of our world, from our homes to the infrastructure that provides mission-critical services. As the size and complexity of software systems increase, the number and sophistication of software security flaws increase as well. The analysis of these flaws began as a manual approach, but it soon became apparent that a manual approach alone cannot scale, and that tools were necessary to assist human experts in this task, resulting in a number of techniques and approaches that automated certain aspects of the vulnerability analysis process. Recently, DARPA carried out the Cyber Grand Challenge, a competition among autonomous vulnerability analysis systems designed to push the tool-assisted human-centered paradigm into the territory of complete automation, with the hope that, by removing the human factor, the analysis would be able to scale to new heights. However, when the autonomous systems were pitted against human experts it became clear that certain tasks, albeit simple, could not be carried out by an autonomous system, as they require an understanding of the logic of the application under analysis. Based on this observation, we propose a shift in the vulnerability analysis paradigm, from tool-assisted human-centered to human-assisted tool-centered. In this paradigm, the automated system orchestrates the vulnerability analysis process, and leverages humans (with different levels of expertise) to perform well-defined sub-tasks, whose results are integrated in the analysis. As a result, it is possible to scale the analysis to a larger number of programs, and, at the same time, optimize the use of expensive human resources. In this paper, we detail our design for a human-assisted automated vulnerability analysis system, describe its implementation atop an open-sourced autonomous vulnerability analysis system that participated in the Cyber Grand Challenge, and evaluate and discuss the significant improvements that non-expert human assistance can offer to automated analysis approaches.