Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?-猫眼课题宝

DOI：

10.48550/arxiv.2203.08392

发表时间：

2022-03

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Y. Fu;Shunyao Zhang;Shan-Hung Wu;Cheng Wan;Yingyan Lin

视觉转换器（ViTs）最近在神经架构设计中掀起了一股新的浪潮，这要归功于它们在各种视觉任务中的破纪录性能。同时，为了实现将ViT部署到现实世界的视觉应用中的目标，它们对潜在恶意攻击的鲁棒性越来越受到关注。特别是，最近的工作表明，与卷积神经网络（CNN）相比，ViTs对对抗性攻击更鲁棒，并推测这是因为ViTs更专注于捕捉不同输入/特征补丁之间的全局交互，从而提高了对抗性攻击施加的局部扰动的鲁棒性。在这项工作中，我们提出了一个有趣的问题：“与CNN相比，在什么样的扰动下，ViTs变得更脆弱？“在这个问题的驱动下，我们首先对ViTs和CNN在各种现有对抗性攻击下的鲁棒性进行了全面的实验，以了解有利于其鲁棒性的根本原因。基于得出的见解，我们提出了一个专用的攻击框架，称为补丁傻瓜，通过攻击其基本组件（即，单个补丁）与一系列注意力感知优化技术。有趣的是，我们的Patch-Fool框架第一次表明，ViTs不一定比CNN对对抗性扰动更鲁棒。特别是，我们发现，与CNN相比，ViTs更容易受到Patch-Fool攻击，这在广泛的实验中是一致的，而Sparse/Mild Patch-Fool（Patch-Fool的两种变体）的观察结果表明，每个补丁上的扰动密度和强度似乎是影响ViTs和CNN之间鲁棒性排名的关键因素。

Vision transformers (ViTs) have recently set off a new wave in neural architecture design thanks to their record-breaking performance in various vision tasks. In parallel, to fulfill the goal of deploying ViTs into real-world vision applications, their robustness against potential malicious attacks has gained increasing attention. In particular, recent works show that ViTs are more robust against adversarial attacks as compared with convolutional neural networks (CNNs), and conjecture that this is because ViTs focus more on capturing global interactions among different input/feature patches, leading to their improved robustness to local perturbations imposed by adversarial attacks. In this work, we ask an intriguing question:"Under what kinds of perturbations do ViTs become more vulnerable learners compared to CNNs?"Driven by this question, we first conduct a comprehensive experiment regarding the robustness of both ViTs and CNNs under various existing adversarial attacks to understand the underlying reason favoring their robustness. Based on the drawn insights, we then propose a dedicated attack framework, dubbed Patch-Fool, that fools the self-attention mechanism by attacking its basic component (i.e., a single patch) with a series of attention-aware optimization techniques. Interestingly, our Patch-Fool framework shows for the first time that ViTs are not necessarily more robust than CNNs against adversarial perturbations. In particular, we find that ViTs are more vulnerable learners compared with CNNs against our Patch-Fool attack which is consistent across extensive experiments, and the observations from Sparse/Mild Patch-Fool, two variants of Patch-Fool, indicate an intriguing insight that the perturbation density and strength on each patch seem to be the key factors that influence the robustness ranking between ViTs and CNNs.

Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?

文献摘要

相似文献