Using Computational Modeling to Transform Assessments of Creativity in Engineering Design

这个由宾夕法尼亚州立大学、马里兰大学和华盛顿与李大学的研究团队合作的项目专注于衡量本科工程教育中的创造力。创造性思维能力对于STEM领域的成功至关重要，尤其是工程领域，这需要为复杂的问题设计解决方案，而这些问题往往没有单一或“正确”的解决方案。《下一代科学标准》将解决问题和灵活性等创造性思维能力确定为现代STEM教育的核心能力。然而，教育工作者目前没有足够的工具来评估他们课堂上的创造力。为了有效地为STEM劳动力做好准备，迫切需要一种评估工具，教育工作者和研究人员可以使用这些工具来确定STEM教育中培养创造力的有效方法。目前的创造力测试给STEM教育工作者带来了巨大的挑战，包括(面对面)试卷管理，以及可能最有问题的人工评分，要求教师对数千个答案进行计数和编码-这是一个劳动密集型且往往成本高昂的过程，特别是对于资源不足的学校。鉴于学生人数日益多样化，无论种族或民族，提供公平和一致地衡量学生能力的创造力测试，对于STEM教育中的机会公平更加关键。该项目旨在创建一个衡量工程设计创造力的在线平台，教育工作者可以使用该平台来满足所有学生的需求。该工具将允许教育工作者管理一系列工程创造力任务，并自动计算创造力分数。该项目符合ECR计划的意图，即促进“新的教育研究、测量和评估方法的开发、改进和测试”。它涉及ECR的研究轨道，“STEM学习和学习环境研究”，并通过设计包容性的、文化和语言多样化的评估工具，对“扩大STEM领域的参与研究”产生额外影响，评估工具针对在追求STEM学习课程和将英语作为第二语言使用者方面仍未得到充分代表的学生。首先是建立一个大型工程设计评估的在线平台-与工科本科生验证所有平台任务-允许教师和研究人员轻松评估创造力，自动计算创造力指标，并生成可定制的学生报告。第二，将该平台应用于宾夕法尼亚州立大学的一门本科设计课程，该课程包括一个为期3周的创意模块(包括关于工程设计创造力的课程和练习)，以从教师和学生那里获得有价值的平台可用性数据，同时评估一门旨在促进工程设计创造力的有前途的本科课程。该团队将应用计算建模和机器学习方面的最新进展-包括设计草图的主动学习和对创造性问题解决任务的基于文本的响应的分布式语义建模。预计这一方法将简化对创造力的教育评估，产生一种用户友好的技术，在课堂上帮助STEM教育工作者。在这个项目中开发的新的计算工具将促进对创造力、心理测量评估和跨不同领域(不仅仅是工程学)的知识和理解。PI团队还将设计符合文化要求和最小偏见的评估工具-特别是针对越来越多将英语作为第二语言的学生-并与STEM教育工作者合作，最大限度地提高该平台在课堂上的可用性。在线平台和课程材料将公开提供，通过为全国STEM教师和研究人员提供在线资源，促进国家向远程教育和研究的过渡(因当前的大流行而加速)。该项目由NSF的EHR核心研究(ECR)计划支持。ECR计划强调基础STEM教育研究，以产生该领域的基础知识。在基本、广泛和持久的关键领域进行投资：STEM学习和STEM学习环境，扩大STEM的参与，以及STEM劳动力发展。该计划支持积累强有力的证据，为理解、构建理论以解释并建议干预和创新以应对教育中持续存在的挑战的努力提供信息。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This collaborative project from research teams at Pennsylvania State University, University of Maryland, and Washington and Lee University focuses on measuring creativity in undergraduate engineering education. The ability to think creatively is essential for success in STEM fields, particularly engineering, which requires designing solutions to complex problems that often have no single or "correct" solution. The Next Generation Science Standards identify creative thinking skills, such as problem solving and flexibility, as core competencies for modern STEM education. Yet educators are not currently equipped with adequate tools to assess creativity in their classrooms. To effectively prepare the STEM workforce, there is a critical need for assessment tools that educators and researchers can use to identify what works in STEM education to foster creativity. Current creativity tests present significant challenges for STEM educators, including (in-person) paper administration and, perhaps most problematically, manual scoring that requires teachers to count and code thousands of responses—a labor-intensive and often costly process, particularly for under-resourced schools. In light of the increasingly diverse student population, the availability of creativity tests that measure student ability fairly and consistently, regardless of race or ethnicity, is even more critical for equity of opportunity in STEM education. This project seeks to create an online platform for measuring creativity in engineering design that educators can use to cater to the needs of all their students. The tool will allow educators to administer a range of engineering creativity tasks and automatically calculate creativity scores. This project fits the intent of the ECR program to facilitate "the development, refinement, and testing of new education research, measurement, and evaluation methodologies." It addresses the ECR research track, "Research on STEM Learning and Learning Environments," and has additional impacts for "Research on Broadening Participation in STEM Fields" by designing inclusive and culturally and linguistically diverse assessment tools targeted to students who remain underrepresented in the pursuit of STEM courses of study and English as second language speakers.Two aims guide this project. First is to build an online platform for large-scale engineering design assessment — validating all platform tasks with undergraduate engineering students — to allow teachers and researchers to easily assess creativity, automatically compute creativity metrics, and generate customizable student reports. Second is to apply the platform in an undergraduate design course at Penn State that includes a 3-week Creativity Module (with lessons and exercises on creativity in engineering design) to obtain valuable platform usability data from both instructors and students, while evaluating a promising undergraduate course intended to promote creativity in engineering design. The team will apply recent advances in computational modeling and machine learning — including active learning of design sketches and distributional semantic modeling of text-based responses to creative problem solving tasks. It is expected that this approach will streamline educational assessment of creativity, resulting in a user-friendly technology to assist STEM educators in the classroom. The novel computational tools developed in this project will advance knowledge and understanding for creativity psychometric assessment and across different fields (not only engineering). The PI team will also design assessment tools that are culturally responsive and minimally biased — especially for the growing number of students who speak English as a second language — and collaborate with STEM educators to maximize the usability of the platform in their classrooms. The online platform and course materials will be publicly available, facilitating the national transition to remote education and research (accelerated by the current pandemic) by providing online resources for STEM teachers and researchers across the country.This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.