On the Security Risks of Knowledge Graph Reasoning
On the Security Risks of Knowledge Graph Reasoning
复制标题
DOI：
10.48550/arxiv.2305.02383
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Zhaohan Xi;Tianyu Du;Changjiang Li;Ren Pang;S. Ji;Xiapu Luo;Xusheng Xiao;Fenglong Ma;Ting Wa
中科院分区：
文献类型：
--
作者：
Zhaohan Xi;Tianyu Du;Changjiang Li;Ren Pang;S. Ji;Xiapu Luo;Xusheng Xiao;Fenglong Ma;Ting Wa
知识图推理（KGR）——通过大型知识图回答复杂的逻辑查询——代表了一项重要的人工智能任务，涉及一系列应用（例如网络威胁搜寻）。然而，尽管 KGR 越来越受欢迎，但考虑到这种功能在安全关键领域的使用越来越多，KGR 的潜在安全风险在很大程度上尚未被探索，这一点令人担忧。这项工作代表着朝着弥合这一巨大差距迈出了坚实的第一步。我们根据对手的目标、知识和攻击向量将 KGR 的安全威胁系统化。此外，我们还提出了 ROAR，一种可以实例化各种此类威胁的新型攻击。通过对代表性用例（例如医疗决策支持、网络威胁搜寻和常识推理）的实证评估，我们证明 ROAR 可以非常有效地误导 KGR 为目标查询建议预定义的答案，但对非目标查询的影响可以忽略不计。最后，我们探索了针对 ROAR 的潜在对策，包括过滤潜在的中毒知识和使用对抗性增强查询进行训练，这导致了几个有前途的研究方向。
Knowledge graph reasoning (KGR) -- answering complex logical queries over large knowledge graphs -- represents an important artificial intelligence task, entailing a range of applications (e.g., cyber threat hunting). However, despite its surging popularity, the potential security risks of KGR are largely unexplored, which is concerning, given the increasing use of such capability in security-critical domains. This work represents a solid initial step towards bridging the striking gap. We systematize the security threats to KGR according to the adversary's objectives, knowledge, and attack vectors. Further, we present ROAR, a new class of attacks that instantiate a variety of such threats. Through empirical evaluation in representative use cases (e.g., medical decision support, cyber threat hunting, and commonsense reasoning), we demonstrate that ROAR is highly effective to mislead KGR to suggest pre-defined answers for target queries, yet with negligible impact on non-target ones. Finally, we explore potential countermeasures against ROAR, including filtering of potentially poisoning knowledge and training with adversarially augmented queries, which leads to several promising research directions.