EAGER: SaTC-EDU: Instilling a Mindset of Adversarial Thinking into Computer Science Courses Early and Often

人工智能（AI）算法和计算机系统中的安全和设计缺陷可能会使我们的个人信息（包括医疗记录等敏感数据）危险地暴露在外，或者可能产生偏见，使部分人口处于不利地位或受到威胁。在这些安全和设计缺陷造成伤害之前成功发现它们的能力取决于了解计算机系统和算法威胁的合格工程师、研究人员和政策制定者。然而，威胁建模通常只在高级计算机科学课程中教授，这是课程的后期课程，并不是所有学生都选择参加。该项目调查了早期和持续接触威胁建模材料和称为“对抗性思维”的思维模式是否能提高学生识别和应对隐私、网络安全和新人工智能技术挑战的能力。对抗性思维指的是采用对手的视角，他们试图利用系统、算法或模型中的弱点。由此产生的课程材料和研究结果将予以传播，预计这些研究结果将促使计算机科学课程方法的改变。该项目建议开发对抗性思维的材料，并将其整合到布朗大学计算机科学课程的入门、中级和高级课程中。项目团队将衡量学生在每门课程以及跨课程的表现和进步。收集的数据将有助于回答项目的核心研究问题：在计算机科学教育的早期和反复遇到对抗性思维的学生是否表现出更好的识别和解决计算机系统安全和人工智能模型中的威胁和缺陷的能力？该项目将通过教学方法、技能和课程结构建议来影响学术计算机科学教育，帮助学生为新技术的复杂性、风险和机遇做好准备。该项目由安全与可信网络空间（SaTC）计划的一项特别倡议支持，旨在促进网络安全、人工智能和教育领域之间前所未有的合作。SaTC项目与《联邦网络安全研究与发展战略计划》和《国家隐私研究战略》保持一致，旨在保护和维护网络系统日益增长的社会和经济效益，同时确保安全和隐私。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Security and design flaws in artificial intelligence (AI) algorithms and computer systems can leave our personal information, including sensitive data such as medical records, dangerously exposed, or can give rise to biases that disadvantage or threaten parts of the population. The ability to successfully find these security and design flaws before they cause harm depends on qualified engineers, researchers, and policymakers who understand threats to computer systems and algorithms. However, threat-modeling is typically taught only in advanced Computer Science courses, which come late in the curriculum and which not all students elect to take. This project investigates whether earlier and continued exposure to material on threat modeling and a mindset called "adversarial thinking" improves students' ability to recognize and address challenges in privacy, cybersecurity, and new AI technologies. Adversarial thinking refers to adopting the perspective of an adversary who seeks to exploit weaknesses in a system, algorithm, or model. The resulting course materials and findings will be disseminated, and the findings are expected to motivate changes in the approach to computer science curricula.The project proposes to develop material on adversarial thinking and integrate it into courses at the introductory, intermediate, and advanced level of Brown University’s computer science curriculum. The project team will measure students' performance and progression within each course as well as across courses. The data collected will help answer the project’s central research question: do students who encounter adversarial thinking early in and repeatedly throughout their computer science education show improved ability to recognize and address threats and flaws in computer systems security and AI models? The project will impact academic computer science education through pedagogical methods, skills, and recommendations for curricular structures that help prepare students for the complexities, risks, and opportunities of new technologies.This project is supported by a special initiative of the Secure and Trustworthy Cyberspace (SaTC) program to foster new, previously unexplored, collaborations between the fields of cybersecurity, artificial intelligence, and education. The SaTC program aligns with the Federal Cybersecurity Research and Development Strategic Plan and the National Privacy Research Strategy to protect and preserve the growing social and economic benefits of cyber systems while ensuring security and privacy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.