SaTC: CORE: Small: Collaborative: ForensicExaminer: Testbed for Benchmarking Digital Audio Forensic Algorithms

SaTC：核心：小型：协作：ForensicExaminer：数字音频取证算法基准测试平台

• 批准号: 1815724
• 负责人: Khalid Malik
• 金额: $ 19.99万
• 依托单位: Oakland University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-15 至 2023-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815724&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Collaborative; ForensicExaminer

The proliferation of powerful smart-computing devices (e.g., smartphones, surveillance systems) capable of production, editing, analysis, and sharing of multimedia files and associated technological advances have affected almost every aspect of our lives. The use of digital multimedia (images, audio, and video) as evidence is rapidly growing in multiple applications, including legal proceedings and law enforcement. However, forensic audio examiners are facing a new challenge of analyzing evidence containing audio from social networking websites, because audio editing and manipulation tools are both sophisticated and easy to use, increasing the risk of audio forgery. The goal of this project is to develop a framework and methods to support audio forensics examiners in detecting and localizing tampering in audio files, including developing novel algorithms to associate files to specific recording devices; creating methods to detect and estimate the risks of attempts to evade existing forgery detectors; evaluating speaker recognition systems in the presence of audio replay attacks; and collecting a large and diverse dataset of recordings that can be used for benchmarking of existing and future audio forensic analysis tools and techniques. The project also has a significant educational component, consisting of a set of hands-on activities involving media generation, manipulation, and analysis aimed at outreach and broadening participation in science, technology, engineering and mathematics (STEM) disciplines including forensic science, digital signal processing, and statistical data analysis and digital forensics.The project is has four main research thrusts. The first will involve designing effective microphone fingerprint modeling and extraction algorithms tailored for audio forensic applications. The team will leverage microphone calibration methods, statistical signal processing techniques for blind microphone fingerprint estimation, and system identification methods for linking an audio recording to a specific recording device. The second thrust aims to investigate the impact of anti-forensic attacks on existing forgery detectors and replay attacks on speaker recognition systems. The research team will design attack models to perturb the underlying forgery detection feature space and analyze performance of existing and new algorithms under these anti-forensics attacks. The third research effort will be focused on designing new audio forensic analysis algorithms robust to these and other emerging anti-forensic attacks. The team will use manipulation methods for anti-forensic attacks and a game-theory-based framework for attack-aware tamper detection and design new forensic methods based on findings of these activities. The fourth research thrust will aim at developing a first-of-its-kind research commons for audio forensics consisting of benchmarking datasets, algorithms, and tools. The team will collect audio from both controlled settings and crowdsourcing in the wild, and use known audio manipulation, editing, and anti-forensic techniques to generate tampered datasets. The team will design and deploy the benchmarking testbed, ForensicExaminer, using a micro services architecture.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

能够制作、编辑、分析和共享多媒体文件的功能强大的智能计算设备（如智能手机、监控系统）的激增以及相关的技术进步几乎影响了我们生活的方方面面。数字多媒体（图像、音频和视频）作为证据的使用在包括法律诉讼和执法在内的多种应用中迅速增长。然而，法医音频审查员在分析包含社交网站音频的证据时面临着新的挑战，因为音频编辑和处理工具既复杂又易于使用，增加了音频伪造的风险。该项目的目标是开发一个框架和方法，以支持音频取证审查员检测和定位音频文件中的篡改，包括开发将文件与特定录音设备相关联的新算法；创建方法来检测和估计试图逃避现有伪造探测器的风险；评估存在音频重放攻击的说话人识别系统；收集大量不同的录音数据集，用于对现有和未来的音频取证分析工具和技术进行基准测试。该项目还有一个重要的教育组成部分，包括一系列涉及媒体生成、操纵和分析的实践活动，旨在推广和扩大科学、技术、工程和数学（STEM）学科的参与，包括法医学、数字信号处理、统计数据分析和数字法医学。该项目有四个主要的研究重点。首先将涉及设计有效的麦克风指纹建模和提取算法，为音频取证应用量身定制。该团队将利用麦克风校准方法、用于盲麦克风指纹估计的统计信号处理技术，以及用于将音频记录连接到特定记录设备的系统识别方法。第二个重点是研究反取证攻击对现有伪造探测器的影响，以及对说话人识别系统的重放攻击。研究团队将设计攻击模型来干扰底层伪造检测特征空间，并分析现有算法和新算法在这些反取证攻击下的性能。第三项研究工作将集中于设计新的音频取证分析算法，以抵御这些和其他新兴的反取证攻击。该团队将使用操纵方法进行反取证攻击，并使用基于博弈论的框架进行攻击感知篡改检测，并根据这些活动的发现设计新的取证方法。第四个研究重点将旨在为音频取证开发首个同类研究公地，包括基准数据集、算法和工具。该团队将从受控环境和野外众包中收集音频，并使用已知的音频操作、编辑和反取证技术来生成篡改的数据集。该团队将使用微服务架构设计和部署基准测试平台ForensicExaminer。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Towards protecting cyber-physical and IoT systems from single- and multi-order voice spoofing attacks

• DOI: 10.1016/j.apacoust.2021.108283
• 发表时间: 2021-07-17
• 期刊: APPLIED ACOUSTICS
• 影响因子: 3.4
• 作者: Javed, Ali;Malik, Khalid Mahmood;Malik, Hafiz
• 通讯作者: Malik, Hafiz

Voice spoofing detection corpus for single and multi-order audio replays

• DOI: 10.1016/j.csl.2020.101132
• 发表时间: 2021-01-01
• 期刊: COMPUTER SPEECH AND LANGUAGE
• 影响因子: 4.3
• 作者: Baumann, Roland;Malik, Khalid Mahmood;Malik, Hafiz
• 通讯作者: Malik, Hafiz

Voice spoofing detector: A unified anti-spoofing framework

• DOI: 10.1016/j.eswa.2022.116770
• 发表时间: 2022-03-11
• 期刊: EXPERT SYSTEMS WITH APPLICATIONS
• 影响因子: 8.5
• 作者: Javed, Ali;Malik, Khalid Mahmood;Irtaza, Aun
• 通讯作者: Irtaza, Aun

SpoTNet: A spoofing-aware Transformer Network for Effective Synthetic Speech Detection

• DOI: 10.1145/3592572.3592841
• 发表时间: 2023-06
• 期刊: Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation
• 作者: Awais Khan;K. Malik

AVFakeNet: A unified end-to-end Dense Swin Transformer deep learning model for audio–visual​ deepfakes detection

AVFakeNet：用于音频、视觉、深度赝品检测的统一端到端 Dense Swin Transformer 深度学习模型

• DOI: 10.1016/j.asoc.2023.110124
• 发表时间: 2023
• 期刊: Applied Soft Computing
• 影响因子: 8.7
• 作者: Ilyas, Hafsa;Javed, Ali;Malik, Khalid Mahmood
• 通讯作者: Malik, Khalid Mahmood