Image Fakery Detection: Towards a Trace Disentangling and Image Deconstruction Approaches

曝光虚假图像和视频的自动化正在获得越来越多的势头。随着视觉资产的指数增长和越来越复杂的内容编辑工具，确实需要开发框架来支持媒体内容的保管链，以保持其证明价值。揭开图像伪造的面纱仍然非常具有挑战性。在文献中，基于足迹的主动篡改和被动篡改是最常用的方法。活动表单主要分为两个阶段，首先在图像获取期间或外部共享之前在图像中插入一个片段(水印/签名)。在第二阶段，反向过程允许将来源和嵌入的摘录并列和整理在一起。相反，被动方法不需要任何附加信息。它们仅仅依赖于这样一个事实，即篡改的图像可能包含可测量的操纵痕迹。解决双因素问题的双线性模型在不同的领域被广泛采用，例如在黑白图像中分离样式和内容(例如，从书法样式中分离字母形状)。自然图像需要更复杂的多线性模型来处理不同的因素(例如，将面部表情与身份、头部姿势和光照分离)。在这项研究中，我试图设计变分自动编码器(VAE)模型来解决图像组成要素解耦的难题。不同的噪声、伪影和马赛克风格可以通过特定的基于对抗体系结构的模型进行反向恢复，该模型致力于在成对和未成对的图像之间进行风格转换。这些模型很好地诱导出用于风格分离的潜在代码。我的动机是，VAE模型在高阶概率图形框架内提供了神经网络体系结构。我的目标是使用统一的框架直接从图像中学习潜在的表征，目的是从篡改的图像中分离出篡改的轮廓。我的假设是操纵的痕迹(例如，局部噪声分布、伪影、前景/背景、场景照明等)周围的钢化区与图像的其余部分不同。在应用方面，拟议的框架可以在个人、组织和社会层面上找到用途。作为对可信但虚构的媒体的免疫剂，它可以被武装部队用来打击任何削弱军队道德的宣传，被政府用来及时回应国内的大量虚假信息(例如，实时揭露对最后一刻的诽谤竞选活动的即时回复)。它还可以被新闻媒体用来在任何大规模公开传播之前核实资产的完整性。在法庭上，这种适用的解决方案将建议使用数字佐证证据(例如，经核实的监控录像)。它甚至会给证据提供决定性的确定性。

The automation of exposing fake images and videos is gaining more and more momentum. With the exponential growth of visual assets and the more and more sophisticated tools of content editing, there is a real need of developing frameworks to support the chain of custody of media content to preserve its probative value. Unmasking image forgery remains very challenging. In the literature, the active and passive tampering footprints-based methods are the most prevalent. The active forms are mostly two-phases, first they insert a snippet (watermark/signature) in the image during acquisition or before external sharing. In a second phase, a reverse process permits to juxtapose and collate together the origin and the embedded snippet. On the contrary, the passive methods do not need any add-on information. They merely rely on the fact that a doctored image may contain measurable traces of manipulation. Bi-linear models for solving two-factor problems were widely adopted in different areas, such as separating style and content in white and black images (e.g., disentangle alphabet shape from calligraphy styles). Natural images need more complex multilinear models to cope with different factors (e.g., decoupling facial expressions from identity, head pose, and illumination). In this research, I'm seeking to design variational autroencoder (VAE) models to deal with the difficult problem of decoupling the constituent factors of the image. The different noise, artifact and mosaicking styles can be reversely recovered by particular adversarial architecture-based models, which are dedicated to style transferring between paired and unpaired images. These models are good for inducing latent codes for style separation. My motivation is that VAE models offer a neural net architecture within a high-order probabilistic graphical framework. My objective, is to use the unified framework to learn a latent representation directly from the image, aiming to disentangle the tempering profiles from doctored images. My assumption is that the traces of manipulation (e.g., local noise distribution, artifacts, foreground/backgrounds, scene lighting etc.) around the tempered region are different from those of the rest of the image. In terms of applications, the proposed framework could find uses at either individual, organizational, and societal levels. As an immunizer against credible yet fictitious media, it can be used by armed forces for combating any propaganda for sapping the troops' moral, by government to timely respond to domestic massive disinformation (e.g., real-time unmasking for an instant reply to a last-minute defamatory election campaign). It can be also used by the news media to verify the integrity of the assets before any mass public spreading. In courtroom, such applicable solutions would suggest digital corroborating evidence (e.g., verified surveillance video). It would even give a decisional certainty to the proof.