EAGER: DCL: SaTC: Enabling Interdisciplinary Collaboration: Improving Human Discernment of Audio Deepfakes via Multi-level Information Augmentation

该项目通过增强技术和社会语言信息来提高听众对音频深度赝品的辨别力。该项目为跨社会语言学、以人为中心的分析和数据科学的合作研究建立了一条创新途径，并为未来对深度伪造品的分析奠定了基础，这些深度伪造品在跨学科方面具有广泛的相关性，并以人类行为观点为基础。该项目将通过产生见解来解决错误信息的社会挑战，这些见解可以提高听众（尤其是大学生，他们的生活受到技术不可磨灭的影响）评估在线信息的真实性和真实性的能力。该项目更广泛的意义是通过产生见解来解决错误信息的社会挑战，这些见解可以帮助听众做出如何评估他们在网上遇到的信息的真实性和真实性的决定。该项目提高了对深度造假如何参与传播错误信息的理解和建模，并跟踪语言技术如何适应社会危害和/或以不道德的方式使用。拟议的工作将通过利用综合跨学科知识的信息增强来提高听众对音频深度伪造品的辨别力，并将数据增强作为深度伪造品检测的重要工具。该项目的目标是：（1）研究和评估听众对以不同程度的语言复杂性创建的音频深度赝品的看法； （2）研究和评估培训课程对提高听众社会语言感知能力和辨别深度伪造音频内容的能力的效果； (3) 通过训练和语言标签提供的多级时间和语言签名增强音频深度伪造识别能力； (4) 评估增强签名信息对听众对音频深度赝品的感知的影响； (5) 在社会科学和数据科学学生的参与下创建开放获取的在线模块和材料，以提高听众在更广泛的公共范围内对音频线索的辨别力。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project increases listeners’ discernment of audio deepfakes through augmentation of information, both technological and sociolinguistic. This project establishes an innovative pathway for collaborative research across sociolinguistics, human centered analytics, and data science and lays the groundwork for future analyses of deepfakes that are broadly relevant across disciplines, informed by human behavioral perspectives. The project will address the societal challenge of misinformation by generating insights that can increase the ability of listeners – particularly college students, whose lives are indelibly shaped by technology – to evaluate the veracity and authenticity of information online. The project's broader significance is to address the societal challenge of misinformation by generating insights that can help empower listeners to make decisions about how to evaluate the veracity and authenticity of information they encounter online. The project improves understanding and modeling of how deepfakes are involved in spreading misinformation and tracking how language technology is adapted for social harm and/or used in unethical ways. The proposed work will increase listeners’ discernment of audio deepfakes through augmentation of information that draws upon integrated interdisciplinary knowledge and advances data augmentation as an important tool for deepfake detection. The objectives of the project are to: (1) Study and evaluate listener perceptions of audio deepfakes that have been created with varying degrees of linguistic complexity; (2) Study and evaluate the efficacy of training sessions that increase listeners’ sociolinguistic perceptual ability and improve their ability to discern deepfake audio content; (3) Augment the audio deepfake discernment via multi-level temporal and linguistic signatures, informed by training and linguistic labeling; (4) Evaluate the impact of augmented signature information on listener perceptions of audio deepfakes; (5) Create open-access online modules and materials with social science and data science student involvement to improve listeners’ discernment of audio cues on a wider public scale.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.