CIF: Small: End to End Security-Oriented Optimization of Image Acquisition Pipelines

确保数字图像的完整性是我们这个时代最具挑战性的问题之一。数字媒体编辑软件的能力不断增强，催生了一大批令人震惊的操纵例子，包括臭名昭著的DeepFake，它被认为是对隐私、民主和国家安全的迫在眉睫的挑战。到目前为止，提出的许多解决方案都达不到要求，而且可以通过非常简单的后处理而变得无效。特别是，社交网络和照片共享服务应用的强大压缩使现有的身份验证协议变得不可靠。此外，在数码相机的图像处理器中越来越多地采用深度学习和计算摄影，甚至在本地照片认证方面也带来了新的挑战。该项目将使用现代机器学习技术对图像获取和分发工作流程进行面向安全的设计，以确保图像的完整性和来源，从而解决这个由社交媒体驱动的世界中出现的一个问题。该项目旨在解决：1)优化成像管道，以便在最具挑战性的条件下进行可靠的法医分析；2)设计适用于各种认证问题的培训协议；3)优化整个获取和分发工作流程，包括成像处理器和有损压缩。初步实验表明，通过利用分发后取证分析的反馈，可以修改成像管道以促进内容认证。神经成像管道可以学习引入不可察觉的伪影，类似于数字水印，这将显著提高操作检测精度，从45%提高到90%以上。这一奖项反映了NSF的法定使命，通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Ensuring integrity of digital images is one of the most challenging problems of our times. Increasing capabilities of digital media editing software has spawned a torrent of shocking manipulation examples, including the infamous DeepFakes which has been deemed a looming challenge for privacy, democracy, and national security. Many solutions proposed to-date have fallen short and can be rendered ineffective with very simple post-processing. In particular, strong compression applied by social networks and photo sharing services render existing authentication protocols unreliable. Moreover, increasing adoption of deep learning and computational photography in imaging processors in digital cameras creates new challenges even in native photo authentication. This project will use modern machine learning techniques to pursue security-oriented design of image acquisition and distribution workflows to ensure that image integrity and provenance can be assured, thereby addressing an emerging problem in this social-media driven world. This project seeks to tackle: 1) optimization of the imaging pipeline to facilitate reliable forensic analysis in the most challenging conditions; 2) design of training protocols that generalize to various authentication problems; 3) optimization of the entire acquisition and distribution workflow, including both the imaging processor and lossy compression. Preliminary experiments indicate that by exploiting feedback from post-distribution forensic analysis, the imaging pipeline can be modified to facilitate content authentication. A neural imaging pipeline can learn to introduce imperceptible artifacts, akin to digital watermarks, which significantly increase manipulation detection accuracy, from 45% to over 90%. Further gains will result from the explorations on this project along the thrusts noted above.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.