Using Computational Modeling to Transform Assessments of Creativity in Engineering Design

宾夕法尼亚州立大学、马里兰大学、华盛顿和李大学的研究团队合作开展了这个项目，重点是衡量本科工程教育中的创造力。创造性思维能力对于在STEM领域取得成功至关重要，尤其是工程领域，这需要为复杂问题设计解决方案，而这些问题往往没有单一或“正确”的解决方案。《下一代科学标准》将创造性思维技能（如解决问题和灵活性）确定为现代STEM教育的核心能力。然而，教育工作者目前还没有足够的工具来评估课堂上的创造力。为了有效地培养STEM劳动力，教育工作者和研究人员迫切需要评估工具，以确定STEM教育中培养创造力的有效方法。目前的创造力测试给STEM教育工作者带来了巨大的挑战，包括（面对面的）纸张管理，以及可能最成问题的手动评分，这需要教师对数千个回答进行计数和编码——这是一个劳动密集型且往往成本高昂的过程，特别是对于资源不足的学校。鉴于学生群体日益多样化，提供不分种族、公平、一致地衡量学生能力的创造力测试，对于STEM教育的机会公平更为重要。该项目旨在创建一个在线平台，用于测量工程设计中的创造力，教育工作者可以使用该平台来满足所有学生的需求。该工具将允许教育工作者管理一系列工程创造力任务，并自动计算创造力分数。该项目符合ECR计划的意图，即促进“新的教育研究、测量和评估方法的开发、改进和测试”。它解决了ECR研究轨道“STEM学习和学习环境研究”，并通过设计包容性和文化和语言多样性的评估工具，针对那些在追求STEM课程和英语作为第二语言的学生，对“扩大STEM领域参与的研究”产生了额外的影响。这个项目有两个目标。首先是建立一个用于大规模工程设计评估的在线平台——与本科生一起验证所有平台任务——允许教师和研究人员轻松评估创造力，自动计算创造力指标，并生成可定制的学生报告。第二是将该平台应用于宾夕法尼亚州立大学的本科设计课程，该课程包括一个为期3周的创意模块（包括工程设计创意的课程和练习），从教师和学生那里获得有价值的平台可用性数据，同时评估一门旨在促进工程设计创意的本科课程。该团队将应用计算建模和机器学习方面的最新进展，包括设计草图的主动学习和基于文本的分布式语义建模，以创造性地解决问题。预计这种方法将简化创造力的教育评估，从而产生一种用户友好的技术，以帮助STEM教育工作者在课堂上学习。在这个项目中开发的新型计算工具将促进对创造力心理测量评估和不同领域（不仅仅是工程）的知识和理解。PI团队还将设计具有文化响应性和最小偏见的评估工具-特别是对于越来越多的以英语为第二语言的学生-并与STEM教育工作者合作，最大限度地提高平台在课堂上的可用性。在线平台和课程材料将向公众开放，通过为全国STEM教师和研究人员提供在线资源，促进全国向远程教育和研究的过渡（目前的大流行加速了这一进程）。本项目由美国国家科学基金会EHR核心研究（ECR）项目支持。ECR项目强调在该领域产生基础知识的基础STEM教育研究。投资在至关重要、广泛和持久的关键领域：STEM学习和STEM学习环境，扩大STEM参与，以及STEM劳动力发展。该项目支持积累有力的证据，为理解、构建理论进行解释提供依据，并提出干预和创新建议，以应对教育中持续存在的挑战。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This collaborative project from research teams at Pennsylvania State University, University of Maryland, and Washington and Lee University focuses on measuring creativity in undergraduate engineering education. The ability to think creatively is essential for success in STEM fields, particularly engineering, which requires designing solutions to complex problems that often have no single or "correct" solution. The Next Generation Science Standards identify creative thinking skills, such as problem solving and flexibility, as core competencies for modern STEM education. Yet educators are not currently equipped with adequate tools to assess creativity in their classrooms. To effectively prepare the STEM workforce, there is a critical need for assessment tools that educators and researchers can use to identify what works in STEM education to foster creativity. Current creativity tests present significant challenges for STEM educators, including (in-person) paper administration and, perhaps most problematically, manual scoring that requires teachers to count and code thousands of responses—a labor-intensive and often costly process, particularly for under-resourced schools. In light of the increasingly diverse student population, the availability of creativity tests that measure student ability fairly and consistently, regardless of race or ethnicity, is even more critical for equity of opportunity in STEM education. This project seeks to create an online platform for measuring creativity in engineering design that educators can use to cater to the needs of all their students. The tool will allow educators to administer a range of engineering creativity tasks and automatically calculate creativity scores. This project fits the intent of the ECR program to facilitate "the development, refinement, and testing of new education research, measurement, and evaluation methodologies." It addresses the ECR research track, "Research on STEM Learning and Learning Environments," and has additional impacts for "Research on Broadening Participation in STEM Fields" by designing inclusive and culturally and linguistically diverse assessment tools targeted to students who remain underrepresented in the pursuit of STEM courses of study and English as second language speakers.Two aims guide this project. First is to build an online platform for large-scale engineering design assessment — validating all platform tasks with undergraduate engineering students — to allow teachers and researchers to easily assess creativity, automatically compute creativity metrics, and generate customizable student reports. Second is to apply the platform in an undergraduate design course at Penn State that includes a 3-week Creativity Module (with lessons and exercises on creativity in engineering design) to obtain valuable platform usability data from both instructors and students, while evaluating a promising undergraduate course intended to promote creativity in engineering design. The team will apply recent advances in computational modeling and machine learning — including active learning of design sketches and distributional semantic modeling of text-based responses to creative problem solving tasks. It is expected that this approach will streamline educational assessment of creativity, resulting in a user-friendly technology to assist STEM educators in the classroom. The novel computational tools developed in this project will advance knowledge and understanding for creativity psychometric assessment and across different fields (not only engineering). The PI team will also design assessment tools that are culturally responsive and minimally biased — especially for the growing number of students who speak English as a second language — and collaborate with STEM educators to maximize the usability of the platform in their classrooms. The online platform and course materials will be publicly available, facilitating the national transition to remote education and research (accelerated by the current pandemic) by providing online resources for STEM teachers and researchers across the country.This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.