Collaborative Research: Automated Analysis of Constructed Response Concept Inventories to Reveal Student Thinking: Forging a National Network for Innovative Assessment Methods

关于美国高等教育在科学、技术、工程和数学（STEM）学科方面的有效性的大量报告呼吁增加对概念学习的重视，而不是死记硬背。然而，对概念学习的适当评估（测试）（通常称为概念清单或诊断问题集）很少，而且受到以具有成本效益的方式对测试结果进行评分的能力的限制。多项选择评估（选择答案）在高等教育中更为普遍，特别是在班级规模较大的大中型院校，自动评分提供了必要的成本效益。相反，书面回答评估（构建的回答）被广泛认为在揭示学生的实际思维方面更出色，但考虑到人工评分需要的时间和精力，它在实践中是相当罕见的。该项目利用最新的计算机化工具和统计技术，使构建的响应评估更广泛地可用。计算机自动化允许在更多的学生中使用这些更有洞察力的概念问题和测试，从而提高对学生概念学习的理解。项目人员与概念测试工具的开发人员合作，创建测试的构建响应版本，并结合必要的计算机评分工具，最终目标是提供概念思维的计算机自动化评估。该项目是三所主要公立大学的合作项目。

Numerous reports on the effectiveness of U.S. higher education in the Science, Technology, Engineering and Mathematics (STEM) disciplines call for increased emphasis on conceptual learning, rather than rote memorization. Suitable assessments (tests) of conceptual learning (often referred to as concept inventories or diagnostic question clusters), however, are few and are constrained by the ability to score the outcomes of the tests in a cost-effective manner. Multiple-choice assessments (selected responses) are more widespread in higher education, especially at medium to large institutions where class sizes are large, and where automated scoring provides the essential cost-effectiveness. Conversely, written response assessments (constructed responses), which are widely held to be superior at revealing actual student thinking, are quite rare in practice given the time and effort required for manual scoring. This project leverages the latest computerized tools and statistical techniques to make constructed response assessments more broadly available. Computer automation allows the use of these more insightful conceptual questions and tests with much larger numbers of students, thereby providing an enhanced understanding of students' conceptual learning. Project personnel work with developers of conceptual testing instruments to create constructed response versions of the tests coupled with the necessary computerized scoring tools with the eventual goal of providing computer-automated evaluation of conceptual thinking. The project is a collaboration among three major public universities.