EAGER: DCL: SaTC: Enabling Interdisciplinary Collaboration: Improving Human Discernment of Audio Deepfakes via Multi-level Information Augmentation

EAGER：DCL：SaTC：实现跨学科合作：通过多级信息增强提高人类对音频深赝品的识别能力

• 批准号: 2210011
• 负责人: Vandana Janeja
• 金额: $ 29.98万
• 依托单位: University of Maryland Baltimore County
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2210011&HistoricalAwards=false
• 关键词: EAGER; DCL; SaTC; Enabling; Interdisciplinary

This project increases listeners’ discernment of audio deepfakes through augmentation of information, both technological and sociolinguistic. This project establishes an innovative pathway for collaborative research across sociolinguistics, human centered analytics, and data science and lays the groundwork for future analyses of deepfakes that are broadly relevant across disciplines, informed by human behavioral perspectives. The project will address the societal challenge of misinformation by generating insights that can increase the ability of listeners – particularly college students, whose lives are indelibly shaped by technology – to evaluate the veracity and authenticity of information online. The project's broader significance is to address the societal challenge of misinformation by generating insights that can help empower listeners to make decisions about how to evaluate the veracity and authenticity of information they encounter online. The project improves understanding and modeling of how deepfakes are involved in spreading misinformation and tracking how language technology is adapted for social harm and/or used in unethical ways. The proposed work will increase listeners’ discernment of audio deepfakes through augmentation of information that draws upon integrated interdisciplinary knowledge and advances data augmentation as an important tool for deepfake detection. The objectives of the project are to: (1) Study and evaluate listener perceptions of audio deepfakes that have been created with varying degrees of linguistic complexity; (2) Study and evaluate the efficacy of training sessions that increase listeners’ sociolinguistic perceptual ability and improve their ability to discern deepfake audio content; (3) Augment the audio deepfake discernment via multi-level temporal and linguistic signatures, informed by training and linguistic labeling; (4) Evaluate the impact of augmented signature information on listener perceptions of audio deepfakes; (5) Create open-access online modules and materials with social science and data science student involvement to improve listeners’ discernment of audio cues on a wider public scale.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目通过扩大技术和社会语言学的信息来增加听众对音频深层的识别。该项目为跨社会语言学，以人类为中心的分析和数据科学的合作研究建立了创新的途径，并为未来分析的深层分析奠定了基础，这些分析是由人类行为观点所启发的，跨学科广泛相关。该项目将通过产生能够提高听众的能力（尤其是由技术塑造的生活的大学生）来评估信息在线信息的真实性和真实性的能力来解决错误信息的社会挑战。该项目的更广泛的意义是通过产生洞察力来应对失误的社会挑战，这些见解可以帮助授权听众做出有关如何评估他们在线遇到的信息的真实性和真实性的决定。该项目改善了对深层参与传播错误信息的理解和建模，并跟踪语言技术如何以社会伤害和/或以不道德的方式使用。拟议的工作将通过扩大信息来增加听众对音频深击的识别，这些信息借鉴了综合的跨学科知识，并将数据增强作为深层检测的重要工具。该项目的目标是：（1）研究和评估听众对具有不同程度的语言复杂性创造的音频深击的看法； （2）研究和评估培训课程的有效性，以提高听众的社会语言感知能力，并提高其识别Deepfake音频内容的能力； （3）通过培训和语言标签所告知的多级临时和语言标志来增强音频深层识别； （4）评估增强签名信息对听众对音频深击的看法的影响； （5）通过社会科学和数据科学学生参与创建开放访问的在线模块和材料，以提高听众在更广泛的公共规模上辨别音频提示。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准来评估通过评估而被认为是宝贵的。

项目成果

Auto Annotation of Linguistic Features for Audio Deepfake Discernment

• DOI: 10.1609/aaaiss.v2i1.27682
• 发表时间: 2024-01
• 期刊: Proceedings of the AAAI Symposium Series
• 作者: Kifekachukwu Nwosu;Chloe Evered;Zahra Khanjani;Noshaba Bhalli;Lavon Davis;Christine Mallinson;V. P. Janeja

Learning to Listen and Listening to Learn: Spoofed Audio Detection Through Linguistic Data Augmentation

• DOI: 10.1109/isi58743.2023.10297267
• 发表时间: 2023-10
• 期刊: 2023 IEEE International Conference on Intelligence and Security Informatics (ISI)
• 作者: Zahra Khanjani;Lavon Davis;Anna Tuz;Kifekachukwu Nwosu;Christine Mallinson;V. P. Janeja