ATM: Automated Threat Modelling for Enterprise AI-enabled Assets

ATM：针对企业人工智能资产的自动威胁建模

• 批准号: 10097952
• 金额: $ 7.61万
• 依托单位: UNIVERSITY OF SHEFFIELD
• 依托单位国家: 英国
• 项目类别: Collaborative R&D
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10097952
• 关键词: ATM; Automated; Threat; Modelling; Enterprise

In today's AI era, most companies use AI assets incorporating machine learning and deep learning models. In this context, AI assists enterprises in their decision-making process. The estimated cost of building and implementing an AI application is **$50k on average**. It is reported that AI will contribute up to **$15.7 trillion** to the global economy **by 2030**. Due to the AI revolution, there has been an increased dependency on AI infrastructures, especially in sectors like Banking and Finance, Retail, Aviation, Autonomous Industries, Insurance, Robotics, etc. Since AI-enabled technologies are present in almost every sector nowadays, the chances of AI going rogue would create a catastrophic impact if not protected.However, AI-enabled asset manufacturers mainly focus on designing, training and deploying AI-based solutions without considering security requirements. As a result, AI-based attacks have become common over recent years. These attacks can potentially manipulate the decision-making of AI-enabled assets in a way that would be imperceptible to humans. The most common attacks on AI/ML are _**Data Poisoning, Model Hijacking, Adversarial, and Transfer Learning Attacks**._To address the negative impacts such threats can cause on client assets, they must know if their device is vulnerable. In this context, enterprises perform threat modelling to understand the vulnerability of their infrastructures. Manual vulnerability assessment is the most common approach; however, it is neither a practical nor accurate method for analysing AI-enabled assets' vulnerabilities as it cannot extract and understand AI algorithms and data. Hence, autonomous AI-assisted threat modelling can facilitate not only the design of a comprehensive and accurate threat model but also assist with making an appropriate response. However, to the best of our knowledge, there is no autonomous AI-enabled threat-modelling solution to analyse threats against **AI-enabled assets in great depth**. Moreover, existing vulnerability assessment approaches do not ensure the confidentiality of clients' data.In this regard, we propose to develop an AI-assisted **Automated Threat Modelling (ATM)** System that will help detect all the threats related to the AI-enabled asset by generating a Threat Model, providing countermeasures, and prioritising them to mitigate discovered threats.

在当今的人工智能时代，大多数公司使用的是融合了机器学习和深度学习模型的人工智能资产。在这种背景下，人工智能帮助企业进行决策。构建和实现一个AI应用程序的估计成本**平均为5万美元**。据报道，到2030年，人工智能对全球经济的贡献将高达**15.7万亿美元**。由于人工智能革命，人们对人工智能基础设施的依赖程度有所增加，特别是在银行和金融、零售、航空、自主工业、保险、机器人等行业。由于如今人工智能技术几乎存在于每个行业，如果不加以保护，人工智能流氓的可能性将产生灾难性的影响。然而，人工智能资产制造商主要专注于设计、培训和部署基于人工智能的解决方案，而不考虑安全需求。因此，基于人工智能的攻击在最近几年变得常见。这些攻击可能会以人类无法察觉的方式操纵人工智能资产的决策。对AI/ML最常见的攻击是_**数据中毒、模型劫持、对抗性攻击和转移学习攻击**。_为了应对这些威胁可能对客户端资产造成的负面影响，他们必须知道自己的设备是否易受攻击。在这种情况下，企业执行威胁建模以了解其基础设施的脆弱性。人工漏洞评估是最常见的方法；然而，它既不是分析人工智能资产漏洞的实用方法，也不是一种准确的方法，因为它无法提取和理解人工智能算法和数据。因此，自主的人工智能辅助威胁建模不仅可以帮助设计一个全面和准确的威胁模型，而且还可以帮助做出适当的响应。然而，就我们所知，目前还没有自主的启用人工智能的威胁建模解决方案来深入分析针对**启用人工智能的资产**的威胁。此外，现有的漏洞评估方法不能确保客户数据的机密性。在这方面，我们建议开发一个人工智能辅助的**自动威胁建模(ATM)**系统，该系统将通过生成威胁模型、提供对策并确定它们的优先顺序来缓解已发现的威胁，从而帮助检测与人工智能启用的资产相关的所有威胁。