Image Fakery Detection: Towards a Trace Disentangling and Image Deconstruction Approaches

自动曝光虚假图片和视频的势头越来越大。随着视觉资产的指数级增长和越来越复杂的内容编辑工具，确实需要开发框架来支持媒体内容的监管链，以保持其证明价值。揭露图像伪造仍然非常具有挑战性。在文献中，基于足迹的主动和被动篡改方法最为普遍。主动形式通常分为两个阶段，首先在采集或外部共享之前在图像中插入一个片段（水印/签名）。在第二阶段，一个反向过程允许将原始代码和嵌入的代码片段并置并比对。相反，被动方法不需要任何附加信息。他们仅仅依赖于这样一个事实，即被篡改的图像可能包含可测量的操纵痕迹。

The automation of exposing fake images and videos is gaining more and more momentum. With the exponential growth of visual assets and the more and more sophisticated tools of content editing, there is a real need of developing frameworks to support the chain of custody of media content to preserve its probative value. Unmasking image forgery remains very challenging. In the literature, the active and passive tampering footprints-based methods are the most prevalent. The active forms are mostly two-phases, first they insert a snippet (watermark/signature) in the image during acquisition or before external sharing. In a second phase, a reverse process permits to juxtapose and collate together the origin and the embedded snippet. On the contrary, the passive methods do not need any add-on information. They merely rely on the fact that a doctored image may contain measurable traces of manipulation. Bi-linear models for solving two-factor problems were widely adopted in different areas, such as separating style and content in white and black images (e.g., disentangle alphabet shape from calligraphy styles). Natural images need more complex multilinear models to cope with different factors (e.g., decoupling facial expressions from identity, head pose, and illumination). In this research, I'm seeking to design variational autroencoder (VAE) models to deal with the difficult problem of decoupling the constituent factors of the image. The different noise, artifact and mosaicking styles can be reversely recovered by particular adversarial architecture-based models, which are dedicated to style transferring between paired and unpaired images. These models are good for inducing latent codes for style separation. My motivation is that VAE models offer a neural net architecture within a high-order probabilistic graphical framework. My objective, is to use the unified framework to learn a latent representation directly from the image, aiming to disentangle the tempering profiles from doctored images. My assumption is that the traces of manipulation (e.g., local noise distribution, artifacts, foreground/backgrounds, scene lighting etc.) around the tempered region are different from those of the rest of the image. In terms of applications, the proposed framework could find uses at either individual, organizational, and societal levels. As an immunizer against credible yet fictitious media, it can be used by armed forces for combating any propaganda for sapping the troops' moral, by government to timely respond to domestic massive disinformation (e.g., real-time unmasking for an instant reply to a last-minute defamatory election campaign). It can be also used by the news media to verify the integrity of the assets before any mass public spreading. In courtroom, such applicable solutions would suggest digital corroborating evidence (e.g., verified surveillance video). It would even give a decisional certainty to the proof.