权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Mathematics of Adversarial Attacks

对抗性攻击的数学

基本信息

批准号：
EP/V046527/1
负责人：
Desmond Higham
金额：
$ 25.75万
依托单位：
University of Edinburgh
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2021
资助国家：
英国
起止时间：
2021 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FV046527%2F1
关键词：
Mathematics Adversarial Attacks

项目摘要

This proposal is built on two observations:1. Empirical experiments have shown that even the most sophisticated and highly-regarded artificial intelligence (AI) tools can be fooled by carefully constructed examples. For example, given a picture of a dog, we can change the picture in a way that is imperceptible to the human eye but makes the AI system change its mind and categorize the picture as a chicken. Such *adversarial attacks* can be shockingly successful, and they clearly have implications for safety, security and ethics. 2. Although many mathematical scientists are contributing to the exciting and fast-moving body of research in AI and deep learning, the main theoretical focus so far has been on approximation power (can we build systems that satisfy a desired list of properties?) and optimization (what is the best way to fine-tune the network details?).There is an urgent, unmet need for actionable understanding around adversarial attacks: are they inevitable, are they identifiable, and are they generalizable to other forms of attack?This motivates the themes of the proposal: Inevitability, Identifiability, and Escalation.Here are three examples of the types of questions that we will address:A) Is it inevitable that any AI system will be susceptible to adversarial attack (in which case we should assign resources to identifying attacks rather than attempting to eliminate them)?B) Typical modern AI hardware is fast but has low accuracy (e.g., each computation may carry only 3 digits); can such imprecision be exploited by new forms of adversarial attack? C) How secure are AI systems to malicious interventions that, rather than attacking the input data, make covert alterations to the parameters in the system? We will, for the first time, develop and extend highly relevant ideas from the field of mathematics (numerical analysis and approximation theory) to produce concepts and tools that allow us to appreciate fundamental limitations of AI technology, and identify when these limitations are being exposed; thereby contributing to issues of security, interpretability and accountability. The proposal will involve a post-doctoral research assistant, who will gain valuable skills in a high-demand area. Also, because issues of trust, privacy and security are central to this project, public engagement activities are built in to the plans. A key route to creating lasting impact is the development of practical case studies that highlight the theory that we develop. This will involve the creation of computer code that uses industry-standard AI platforms and data sets: it is an activity that requires specialist skills in coding and data science, and a qualified software engineer will be employed for this task. Overall, the ideas emerging from this project will transform our understanding of AI systems by using currently overlooked techniques from computational mathematics. Furthermore, by showing that there are challenges at the heart of AI that can be tackled by computational and applied mathematicians, we plan to transform the scale and quality of research interaction at this important mathematics-computer science interface.

这一建议是建立在两个观察：1。经验性实验表明，即使是最复杂和最受重视的人工智能（AI）工具也可能被精心构建的示例所愚弄。例如，给定一张狗的图片，我们可以以人眼无法察觉的方式改变图片，但使AI系统改变主意并将图片归类为鸡。这种对抗性攻击可能会取得令人震惊的成功，它们显然会对安全、安保和道德产生影响。2.尽管许多数学科学家正在为人工智能和深度学习领域令人兴奋和快速发展的研究做出贡献，但到目前为止，主要的理论焦点一直是近似能力（我们能否构建满足所需属性列表的系统？）和优化（微调网络细节的最佳方法是什么？）。有一个紧迫的，未满足的需要，围绕敌对性攻击的可操作的理解：他们是不可避免的，他们是可识别的，他们是一般化的其他形式的攻击？这激发了提案的主题：不可避免性，可识别性和升级。下面是我们将解决的问题类型的三个例子：A）任何AI系统都不可避免地容易受到对抗性攻击吗（在这种情况下，我们应该分配资源来识别攻击，而不是试图消除它们）？B）典型的现代AI硬件速度快但精度低（例如，每次计算可能只携带3位数）;这种不精确性会被新形式的对抗性攻击利用吗？C）人工智能系统对恶意干预的安全性有多高，这些恶意干预不是攻击输入数据，而是对系统中的参数进行隐蔽更改？我们将首次开发和扩展数学领域（数值分析和近似理论）中高度相关的思想，以产生概念和工具，使我们能够理解人工智能技术的基本局限性，并确定这些局限性何时暴露;从而有助于解决安全性，可解释性和问责制问题。该提案将涉及一名博士后研究助理，他将在高需求领域获得宝贵的技能。此外，由于信任、隐私和安全问题是该项目的核心，公众参与活动也纳入了计划。创造持久影响的一个关键途径是开发突出我们开发的理论的实际案例研究。这将涉及使用行业标准的人工智能平台和数据集创建计算机代码：这是一项需要编码和数据科学专业技能的活动，并将聘请一名合格的软件工程师来完成这项任务。总的来说，这个项目产生的想法将通过使用目前被忽视的计算数学技术来改变我们对人工智能系统的理解。此外，通过证明人工智能的核心存在可以由计算和应用数学家解决的挑战，我们计划在这个重要的数学-计算机科学接口上改变研究互动的规模和质量。