EAGER: DCL: SaTC: Enabling Interdisciplinary Collaboration: Improving Human Discernment of Audio Deepfakes via Multi-level Information Augmentation

该项目通过技术和社会语言学信息的增强来提高听众对音频深度伪造的识别力。该项目为跨社会语言学、以人为本的分析和数据科学的合作研究建立了一条创新途径，并为未来分析跨学科广泛相关的deepfakes奠定了基础。该项目将通过产生见解来解决错误信息的社会挑战，这些见解可以提高听众的能力-特别是大学生，他们的生活被技术不可磨灭地塑造-以评估在线信息的真实性和真实性。该项目更广泛的意义是通过产生见解来解决错误信息的社会挑战，这些见解可以帮助听众就如何评估他们在网上遇到的信息的真实性和真实性做出决定。该项目提高了对deepfakes如何参与传播错误信息的理解和建模，并跟踪语言技术如何适应社会危害和/或以不道德的方式使用。拟议的工作将通过利用综合跨学科知识的信息增强来提高听众对音频深度伪造的识别力，并将数据增强作为深度伪造检测的重要工具。该项目的目标是：（1）研究和评估听众对具有不同程度语言复杂性的音频deepfake的感知;（2）研究和评估培训课程的有效性，以提高听众的社会语言感知能力，并提高他们辨别deepfake音频内容的能力;（3）通过训练和语言标记告知，经由多级时间和语言签名来增强音频深度伪造识别;（4）评估增强的签名信息对收听者对音频深度伪造的感知的影响;（5）在社会科学和数据科学学生的参与下，创建开放获取的在线模块和材料，以提高听众在更广泛的公众范围内对音频线索的辨别力。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project increases listeners’ discernment of audio deepfakes through augmentation of information, both technological and sociolinguistic. This project establishes an innovative pathway for collaborative research across sociolinguistics, human centered analytics, and data science and lays the groundwork for future analyses of deepfakes that are broadly relevant across disciplines, informed by human behavioral perspectives. The project will address the societal challenge of misinformation by generating insights that can increase the ability of listeners – particularly college students, whose lives are indelibly shaped by technology – to evaluate the veracity and authenticity of information online. The project's broader significance is to address the societal challenge of misinformation by generating insights that can help empower listeners to make decisions about how to evaluate the veracity and authenticity of information they encounter online. The project improves understanding and modeling of how deepfakes are involved in spreading misinformation and tracking how language technology is adapted for social harm and/or used in unethical ways. The proposed work will increase listeners’ discernment of audio deepfakes through augmentation of information that draws upon integrated interdisciplinary knowledge and advances data augmentation as an important tool for deepfake detection. The objectives of the project are to: (1) Study and evaluate listener perceptions of audio deepfakes that have been created with varying degrees of linguistic complexity; (2) Study and evaluate the efficacy of training sessions that increase listeners’ sociolinguistic perceptual ability and improve their ability to discern deepfake audio content; (3) Augment the audio deepfake discernment via multi-level temporal and linguistic signatures, informed by training and linguistic labeling; (4) Evaluate the impact of augmented signature information on listener perceptions of audio deepfakes; (5) Create open-access online modules and materials with social science and data science student involvement to improve listeners’ discernment of audio cues on a wider public scale.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.