How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals
How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals
复制标题
DOI：
10.1109/ijcb48548.2020.9304909
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
U. Ciftci;Ilke Demir;L. Yin
中科院分区：
文献类型：
--
作者：
U. Ciftci;Ilke Demir;L. Yin
虚假肖像视频生成技术通过用于政治宣传、名人模仿、伪造证据和其他身份相关操纵的逼真深度伪造技术，对社会构成了新的威胁。在这些生成技术之后，一些检测方法由于其高分类精度也被证明是有用的。然而，几乎没有花任何努力来追查深度造假的来源。我们提出了一种方法，不仅可以将深度赝品与真实视频分开，还可以发现深度赝品背后的特定生成模型。一些基于纯深度学习的方法尝试使用 CNN 对深度赝品进行分类，其中它们实际上学习了生成器的残差。我们相信这些残差包含更多信息，我们可以通过将它们与生物信号分开来揭示这些操纵伪影。我们的主要观察结果表明，生物信号中的时空模式可以被视为残差的代表性投影。为了证明这一观察结果的合理性，我们从真实和虚假视频中提取 PPG 细胞，并将其输入最先进的分类网络，以检测每个视频的生成模型。我们的结果表明，我们的方法可以以 97.29% 的准确率检测假视频，并且可以以 93.39% 的准确率检测源模型。
Fake portrait video generation techniques have been posing a new threat to the society with photorealistic deep fakes for political propaganda, celebrity imitation, forged evidences, and other identity related manipulations. Following these generation techniques, some detection approaches have also been proved useful due to their high classification accuracy. Nevertheless, almost no effort was spent to track down the source of deep fakes. We propose an approach not only to separate deep fakes from real videos, but also to discover the specific generative model behind a deep fake. Some pure deep learning based approaches try to classify deep fakes using CNNs where they actually learn the residuals of the generator. We believe that these residuals contain more information and we can reveal these manipulation artifacts by disentangling them with biological signals. Our key observation yields that the spatiotemporal patterns in biological signals can be conceived as a representative projection of residuals. To justify this observation, we extract PPG cells from real and fake videos and feed these to a state-of-the-art classification network for detecting the generative model per video. Our results indicate that our approach can detect fake videos with 97.29% accuracy, and the source model with 93.39% accuracy.