Development and Validation of Assessments for Industry-Valued Professional and Technical Learning Outcomes in Engineering Education

为了培养对行业有更高价值的工程毕业生，课程改革需要对知识、技能和能力（KSA）进行有效评估，如项目管理、在多学科团队中的表现、批判性思维、解决开放式问题、企业家思维、与不同受众沟通、承担计算风险以及评估商业和技术权衡。本项目侧重于在高级顶点设计课程中进行KSA评估，让学生在实际项目经验中获得技术和专业技能。项目团队正在定义所需的专业技能并确定优先级；开发、测试和验证在不同环境下衡量学生个人成就的评估；并通过研讨会、会议演讲和重点交流工具（包括Capstone设计会议和基于互联网的Capstone设计中心）传播经过验证的评估。经过验证的评估工具将适用于所有学科、项目类型、团队配置和学生背景的工程顶点设计课程。正在根据不同学生群体的投入和测试开发工具，特别关注女性和少数民族群体。仪器的心理测量分析将涉及使用经典测试理论（CTT）和项目反应理论（IRT）心理测量分析。具体而言，将使用经典测试理论来分析项目属性（项目难度和判别指标）以及测试属性（测试信度和判别指标）。CTT分析的结果将允许团队验证(a)项目是否能够在测试它们所要测试的内容时发挥良好的作用；(b)如果没有，哪些项目可能需要进一步检查。一个多样化的实践者社区正在为评估的持续改进和实施做出贡献，以使它们获得广泛的采用，指导课程变化，并通过接受更好教育的工程师进入全球劳动力市场来造福社会。

To educate engineering graduates of higher value to industry upon graduation, curriculum reform requires effective assessment of knowledge, skills and abilities (KSA) such as project management, performance on multidisciplinary teams, critical thinking, addressing open-ended problems, thinking entrepreneurially, communicating with diverse audiences, taking calculated risks, and evaluating business and technical trade-offs. This project focuses on KSA assessment in senior capstone design courses where students gain technical and professional skills in realistic project experiences. The project team is defining and prioritizing required professional skills; developing, testing, and validating assessments that measure individual student achievement in diverse settings; and disseminating proven assessments through workshops, conference presentations, and focused communication vehicles including the Capstone Design Conference and an Internet-based Capstone Design Hub. Validated assessment tools will apply to engineering capstone design courses of all disciplines, project types, team configurations, and student backgrounds. Instruments are being developed with input from and testing with diverse student populations, giving specific attention to female and minority populations. Psychometric analysis of the instruments will involve the use of Classical Test Theory (CTT) and Item Response Theory (IRT) psychometric analysis. Specifically, the Classical Test Theory will be used to analyze item properties (item difficulty and discrimination indices) as well as test properties (test reliability and discrimination indices). Results from the CTT analysis will allow the team to verify (a) whether items can collectively function well in testing what they are purported to test; and (b) if not, which items may warrant further inspection. A diverse community of practitioners is contributing to the ongoing refinement and implementation of the assessments in order for them to gain broad adoption, guide curriculum change, and benefit society through better educated engineers entering the global workforce.