EAGER: SaTC-EDU: Teaching Security in Undergraduate Artificial Intelligence Courses Using Transparency and Contextualization

这个项目将探索如何向计算机科学本科生传授使用人工智能(AI)的系统中的安全知识。这对于培养一支了解稳健和值得信赖的人工智能的劳动力来说非常重要。其目的是设计一门人工智能课程，以培养一种安全思维，以识别可能造成伤害的漏洞，无论是通过恶意行为者的攻击，还是通过永久或放大社会偏见。教育方法将侧重于透明度和情境化。透明度包括让学生能够接触到系统的内部工作原理，这样他们就可以了解系统结构的哪些方面导致了它的漏洞。情境化涉及将人工智能技术置于现实世界环境中，以了解其具体的安全影响。语境化是教授传统安全主题的基础。例如，在谷歌地图中，访问个人位置数据可以服务于合法目的，但在游戏中通常是可疑行为。同一段代码可以在每种情况下使用，但其合法性取决于其更广泛的上下文。该团队将对将透明度和情境化融入本科人工智能课程进行研究，并开发教学材料和评估工具。由于人工智能的安全是计算机科学教育研究的一个新领域，主要目标是开发初步的教学和评估设计，在适合本科生的水平上整合透明度和情景。目标是开发概念验证教学材料和技术，以及在本科人工智能课程中对安全概念和技能的评估。教学将针对四种学习目标进行设计。学生应该：(1)知道人工智能系统可能造成危害，并不能幸免于攻击；(2)能够解释漏洞的来源；(3)能够识别特定系统中的漏洞，这可能包括攻击它；(4)能够通过修改人工智能系统来防御它，以减轻威胁。该团队将在现有课程中确定具有安全影响的人工智能主题。该团队将创建说明具体安全问题的任务，并与人工智能和安全专家一起进行认知任务分析，看看他们是如何处理这些问题的。这个过程将产生最初的学习目标。该团队将对参加过本科人工智能课程的学生进行一项关于这些目标的评估调查，以建立一个基线知识水平，并引发潜在的误解。根据学生所拥有的基础和学习目标，团队将设计初始指导，通过一对一的思考和与学生参与者的小组辅导会议迭代设计。该团队将在一项受控实验中测试这一指令，将人工智能加安全材料与仅使用人工智能的材料进行比较，使用前后测试来衡量学习。最后，在本科课程中使用设计的教学的研究将说明它在典型环境中是如何工作的。该项目得到了安全和值得信赖的网络空间(SATC)计划的一项特别倡议的支持，该计划旨在促进网络安全、人工智能和教育领域之间新的、以前从未探索过的合作。SATC计划与联邦网络安全研究和发展战略计划和国家隐私研究战略保持一致，以保护和维护网络系统日益增长的社会和经济效益，同时确保安全和隐私。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project will explore how to teach undergraduate computer science students about security in systems that use artificial intelligence (AI). This is important for educating a workforce that is knowledgeable about robust and trustworthy AI. The aim is to design an AI curriculum that will foster a security mindset for identifying vulnerabilities that could cause harm, whether through attacks by a malicious actor, or through perpetuating or amplifying social biases. The educational approach will focus on transparency and contextualization. Transparency involves making the inner workings of a system accessible to students, so they can understand which aspects of the system’s construction lead to its vulnerabilities. Contextualization involves situating AI techniques in real-world environments to understand their specific security implications. Contextualization is fundamental for teaching conventional security topics. For instance, accessing personal location data can serve a legitimate purpose in Google Maps, but is typically suspicious behavior in a game. The same piece of code may be used in each case, but its legitimacy is determined by its broader context. The team will conduct research on, and develop instructional materials and assessment tools for, integrating transparency and contextualization into the undergraduate AI curriculum. Since security in AI is a new area within computer science education research, the main goal is to develop initial designs for instruction and assessment that integrate transparency and contextualization at a level appropriate for undergraduates. The goal is to develop proof-of-concept instructional materials and techniques, and assessments for security concepts and skills in undergraduate AI courses. Instruction will be designed for four kinds of learning objectives. Students should: (1) know that AI systems can cause harms and are not immune to attacks; (2) be able to explain sources of vulnerabilities; (3) be able to identify vulnerabilities in a specific system, which could include attacking it; and (4) be able to defend an AI system by modifying it to mitigate threats. The team will identify AI topics in existing curricula that have security implications. The team will create tasks that illustrate the concrete security issues and conduct cognitive task analyses with experts in AI and security to see how they approach those problems. This process will yield the initial learning goals. The team will conduct an assessment survey on those goals with students who have taken the undergraduate AI course, to establish a baseline level of knowledge and elicit potential misconceptions. Based on the foundation that students have and the learning goals, the team will design initial instruction, iterating on the design through one-on-one think-alouds and small-group tutoring sessions with student participants. The team will test the instruction in a controlled experiment, comparing the AI plus security materials to AI-only materials, using pre- and post-tests to measure learning. Finally, a study using the designed instruction in an undergraduate course will illustrate how it works in a typical setting. This project is supported by a special initiative of the Secure and Trustworthy Cyberspace (SaTC) program to foster new, previously unexplored, collaborations between the fields of cybersecurity, artificial intelligence, and education. The SaTC program aligns with the Federal Cybersecurity Research and Development Strategic Plan and the National Privacy Research Strategy to protect and preserve the growing social and economic benefits of cyber systems while ensuring security and privacy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.