PROVES: Establishing Image Provenance using Semantic Signatures
PROVES: Establishing Image Provenance using Semantic Signatures
复制标题
DOI：
10.1109/wacv51458.2022.00307
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Mingyang Xie;Manav Kulshrestha;Shaojie Wang;Jinghan Yang;Ayan Chakrabarti;Ning Zhang;Yevgeniy Vorobeychik
中科院分区：
文献类型：
--
作者：
Mingyang Xie;Manav Kulshrestha;Shaojie Wang;Jinghan Yang;Ayan Chakrabarti;Ning Zhang;Yevgeniy Vorobeychik
现代人工智能工具，如生成对抗网络，已经改变了我们创建和修改视觉数据的能力，并产生了逼真的结果。然而，这些进步的有害副作用之一是在操纵视觉数据中的信息方面出现了邪恶的用途，例如通过使用深度伪造。我们提出了一种新的架构来保留图像中语义信息的来源，使它们不容易受到深度假攻击。我们的体系结构包括语义签名和验证步骤。我们应用这种架构来验证两种类型的语义信息：个人身份（面孔）和照片是在室内还是室外拍摄的。验证考虑了一系列常见的图像变换，例如平移、缩放、裁剪和小旋转，并拒绝对抗性变换，例如对抗性扰动，或者在人脸验证的情况下，交换的人脸。实验表明，在图像中人脸来源的情况下，我们的方法对黑盒对抗变换（被拒绝）和良性变换（被接受）都具有鲁棒性，很少有假阴性和假阳性。另一方面，背景验证容易受到黑盒对抗示例的影响，但在对抗训练后变得更加健壮。
Modern AI tools, such as generative adversarial networks, have transformed our ability to create and modify visual data with photorealistic results. However, one of the deleterious side-effects of these advances is the emergence of nefarious uses in manipulating information in visual data, such as through the use of deep fakes. We propose a novel architecture for preserving the provenance of semantic information in images to make them less susceptible to deep fake attacks. Our architecture includes semantic signing and verification steps. We apply this architecture to verifying two types of semantic information: individual identities (faces) and whether the photo was taken indoors or outdoors. Verification accounts for a collection of common image transformation, such as translation, scaling, cropping, and small rotations, and rejects adversarial transformations, such as adversarially perturbed or, in the case of face verification, swapped faces. Experiments demonstrate that in the case of provenance of faces in an image, our approach is robust to black-box adversarial transformations (which are rejected) as well as benign transformations (which are accepted), with few false negatives and false positives. Background verification, on the other hand, is susceptible to black-box adversarial examples, but be-comes significantly more robust after adversarial training.