SFS-Capacity: Collaborative: Validation of Concept Assessment Tools for Cybersecurity

马里兰大学巴尔的摩分校（UMBC）的该项目旨在评估两种网络安全教育评估工具：网络安全概念清单（CCI）和网络安全课程评估（CCA）。CCI评估网络安全第一门课程的教学质量；CCA评估的是大学课程对毕业生从事网络安全职业的准备程度。评估将包括学生访谈、专家评论和统计分析，以检查这些工具的有效性。此外，该项目将通过比较学生与教师的表现，初步了解教育方法的有效性。教学实践。目前，还没有一个严谨的、基于研究的方法来衡量网络安全教学的质量。经过验证的评估工具，如UMBC正在开发的工具是必不可少的，这样教育工作者就有了可靠的方法来辨别提高学生准备工作的努力是否成功。该验证研究将完成更广泛的网络安全评估工具项目的重要部分，该项目将为评估和评估教育实践提供严格的、基于证据的工具。为CCI的有效性提供证据？s和CCA？S测量学生对网络安全的概念理解，该项目将完成每个评估工具的六个任务：(1)对学生在回答问题草稿时进行至少30次认知访谈，并使用结果来提高问题的有效性；(2)收集并使用至少20位网络安全或网络安全教育专家对问题草案的反馈；(3)对至少200名学生使用草稿工具，并使用心理测量学方法对结果进行分析；（四）根据任务一至三项对题稿进行修改；(5)对至少1000名学生实施修订后的工具，并对结果进行心理测量分析；(6)在四次网络安全教育会议上举办研讨会，以提高对这些工具的认识，并寻求对这些工具的额外反馈。通过遵循完善严格的cci开发方法，该项目将帮助网络安全教育界开发和改进课程、教学方法和材料。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The project at the University of Maryland, Baltimore County (UMBC) aims to evaluate two cybersecurity educational assessment tools: the Cybersecurity Concept Inventory (CCI) and the Cybersecurity Curriculum Assessment (CCA). The CCI assesses the quality of instruction of any first course in cybersecurity; the CCA assesses how well a college curriculum prepares graduates for a career in cybersecurity. The evaluation will consist of student interviews, expert reviews, and statistical analyses to examine the efficacy of these tools. In addition, the project will provide initial insights into the efficacy of educational approaches by comparing student performance relative to their instructors? teaching practices. At present, there is not a rigorous, research-based method for measuring the quality of cybersecurity instruction. Validated assessment tools such as the ones in development at UMBC are essential so that educators have trusted methods for discerning whether efforts to improve student preparation are successful. This validation study will complete an essential part of the broader Cybersecurity Assessment Tools project, which will provide rigorous, evidence-based instruments for assessing and evaluating educational practices. To develop evidence for the validity of the CCI?s and CCA?s measurement of student conceptual understanding of cybersecurity, the project will complete six tasks for each assessment tool: (1) conduct at least thirty cognitive interviews of students thinking aloud while answering the draft questions, and use results to improve the validity of the questions; (2) collect and use feedback on the draft questions from at least twenty experts in cybersecurity or cybersecurity education; (3) administer the draft tool to at least 200 students and use psychometric methods of analysis on the results; (4) revise the draft questions in light of tasks 1-3; (5) administer the revised tool to at least 1,000 students and use psychometric analysis on the results; and (6) hold workshops at four cybersecurity education meetings to promote awareness of, and to seek additional feedback on the tools. By following well-established rigorous methods for developing CCIs, this project will help the cybersecurity educational community to develop and improve curricula and instructional methods and materials.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.