Knowledge Guided Two-player Reinforcement Learning for Cyber Attacks and Defenses
Knowledge Guided Two-player Reinforcement Learning for Cyber Attacks and Defenses
复制标题
DOI：
10.1109/icmla55696.2022.00213
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Aritran Piplai;M. Anoruo;Kayode Fasaye;A. Joshi;Timothy W. Finin;Ahmad Ridley
中科院分区：
文献类型：
--
作者：
Aritran Piplai;M. Anoruo;Kayode Fasaye;A. Joshi;Timothy W. Finin;Ahmad Ridley
网络防御演习是了解组织在面临网络威胁时的技术能力的重要途径。从这些练习中获得的信息往往会导致找到看不见的方法来利用组织中的漏洞。这些通常会导致更好的防御机制，可以对抗以前未知的漏洞。随着网络战仿真平台的发展，我们可以生成一个防御演习环境，并训练基于强化学习（RL）的自主代理来攻击模拟环境所描述的系统。在本文中，我们描述了一个基于两个玩家游戏的RL环境，同时提高了攻击者和防御者代理的性能。我们通过使用网络安全知识图中关于攻击和缓解步骤的专家知识来指导RL代理，进一步加速RL代理的收敛。我们已经实施并将我们提出的方法集成到CyberBattleSim系统中。
Cyber defense exercises are an important avenue to understand the technical capacity of organizations when faced with cyber-threats. Information derived from these exercises often leads to finding unseen methods to exploit vulnerabilities in an organization. These often lead to better defense mechanisms that can counter previously unknown exploits. With recent developments in cyber battle simulation platforms, we can generate a defense exercise environment and train reinforcement learning (RL) based autonomous agents to attack the system described by the simulated environment. In this paper, we describe a two-player game-based RL environment that simultaneously improves the performance of both the attacker and defender agents. We further accelerate the convergence of the RL agents by guiding them with expert knowledge from Cybersecurity Knowledge Graphs on attack and mitigation steps. We have implemented and integrated our proposed approaches into the CyberBattleSim system.