EAGER: SaTC-EDU: Instilling a Mindset of Adversarial Thinking into Computer Science Courses Early and Often

人工智能（AI）算法和计算机系统中的安全和设计缺陷可能会使我们的个人信息（包括医疗记录等敏感数据）危险地暴露出来，或者可能会引起对部分人群不利或威胁他们的偏见。在造成伤害之前成功找到这些安全和设计缺陷的能力取决于了解计算机系统和算法威胁的合格工程师、研究人员和政策制定者。 然而，威胁建模通常只在高级计算机科学课程中教授，这些课程在课程中较晚，并且不是所有学生都选择参加。该项目调查了早期和持续接触有关威胁建模的材料和称为“对抗性思维”的心态是否可以提高学生识别和解决隐私，网络安全和新AI技术挑战的能力。对抗性思维是指采用对手的观点，试图利用系统，算法或模型中的弱点。由此产生的课程材料和调查结果将予以传播，预计调查结果将推动计算机科学课程方法的变革，该项目提议编写关于对抗性思维的材料，并将其纳入布朗大学计算机科学课程的入门、中级和高级课程。项目团队将衡量学生在每门课程以及跨课程的表现和进步。收集的数据将有助于回答该项目的核心研究问题：在计算机科学教育早期和整个过程中反复遇到对抗性思维的学生是否表现出识别和解决计算机系统安全和人工智能模型中的威胁和缺陷的能力有所提高？该项目将通过教学方法、技能和课程结构建议来影响学术计算机科学教育，帮助学生为新技术的复杂性、风险和机遇做好准备。该项目得到了安全可信网络空间（SaTC）计划的特别倡议的支持，旨在促进网络安全、人工智能、和教育SATC计划与联邦网络安全研究和发展战略计划和国家隐私研究战略保持一致，以保护和维护网络系统日益增长的社会和经济效益，同时确保安全和隐私。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Security and design flaws in artificial intelligence (AI) algorithms and computer systems can leave our personal information, including sensitive data such as medical records, dangerously exposed, or can give rise to biases that disadvantage or threaten parts of the population. The ability to successfully find these security and design flaws before they cause harm depends on qualified engineers, researchers, and policymakers who understand threats to computer systems and algorithms. However, threat-modeling is typically taught only in advanced Computer Science courses, which come late in the curriculum and which not all students elect to take. This project investigates whether earlier and continued exposure to material on threat modeling and a mindset called "adversarial thinking" improves students' ability to recognize and address challenges in privacy, cybersecurity, and new AI technologies. Adversarial thinking refers to adopting the perspective of an adversary who seeks to exploit weaknesses in a system, algorithm, or model. The resulting course materials and findings will be disseminated, and the findings are expected to motivate changes in the approach to computer science curricula.The project proposes to develop material on adversarial thinking and integrate it into courses at the introductory, intermediate, and advanced level of Brown University’s computer science curriculum. The project team will measure students' performance and progression within each course as well as across courses. The data collected will help answer the project’s central research question: do students who encounter adversarial thinking early in and repeatedly throughout their computer science education show improved ability to recognize and address threats and flaws in computer systems security and AI models? The project will impact academic computer science education through pedagogical methods, skills, and recommendations for curricular structures that help prepare students for the complexities, risks, and opportunities of new technologies.This project is supported by a special initiative of the Secure and Trustworthy Cyberspace (SaTC) program to foster new, previously unexplored, collaborations between the fields of cybersecurity, artificial intelligence, and education. The SaTC program aligns with the Federal Cybersecurity Research and Development Strategic Plan and the National Privacy Research Strategy to protect and preserve the growing social and economic benefits of cyber systems while ensuring security and privacy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.