Collaborative Research: Integrating Models of Cognition and Measurement with Conceptual Knowledge: Establishing the Validity and Diagnostic Capability of Concept Inventories

这个合作研究项目实施了一个全面的，多学科的方法来设计和验证的概念库存（CI）。其目的是促进和加强STEM教育中概念清单在形成性和终结性评估中的有效使用。为了实现这一目标，该项目将侧重于一个特定的概念清单，热传输概念清单（TTCI）。数据将从不同的本科生样本中收集。研究将开发一套明确的诊断测量和报告的理解方面，将收集多层次的经验数据的学生表现的TTCI（包括学生协议和访谈研究，大规模的数据心理测量建模），将修订项目和方面的基础上，这些经验数据分析，并将收集进一步的数据修订后的工具。TTCI的数据将根据其在科学和工程关键领域中所利用的基本概念知识和技能进行分析。学生推理和表现的其他经验数据将采用现代心理测量分析技术进行分析，重点是诊断建模。预期的研究成果是：（1）通过网络提供改进的TTCI，并附有供教师使用的解释工具;（2）对TTCI的评估有效性和诊断能力进行深入的多层次分析;（3）对CI的可能形成性和总结性用途进行全面盘点，包括STEM从业者需要什么来充分利用CI;以及（4）用于设计、开发、评估、改进和/或实施其他STEM领域的CI的研发框架。总的来说，这项研究将有助于更好地了解学生的学习和更真实的诊断和形成性评估，从而更好地学习和教授STEM中的重要概念。TTCI的使用会引起学生在热和运输工程科学方面的误解，如流体力学，传热学和热力学。TTCI特别关注被认为对本科工程专业学生重要但困难的概念。因此，拟议中的研究很可能会导致TTCI和其他CI的形成课堂的目的，以及各种总结性的目的扩大使用。该项目是心理测量建模和方法的扩展，将诊断模型应用于CI，并在科学和工程的关键概念的测量和评估领域的扩展评估有效性。

This collaborative research project implements a comprehensive, multidisciplinary approach to the design and validation of concept inventories (CIs). The purpose is to braoden and enhance the effective use of concept inventories in STEM education for formative and summative assessment. To accomplish this goal the project will focus on a specific concept inventory, the Thermal Transport Concept Inventory (TTCI). The data will be collected from the varied undergarduate student samples. The research would develop an explicit set of facets of understanding for diagnostic measurement and reporting, would gather multi-level empirical data of student performance on the TTCI (including student protocol and interview studies, and large scale data for psychometric modeling), would revise both the items and facets based upon these empirical data analyses, and would gather further data on the revised instrument. The data on TTCI will be analyzed in terms of the underlying conceptual knowledge and skills that it taps in critical areas of science and engineering. Additional empirical data on student reasoning and performance will be analyzed applying modern psychometric analysis techniques focused on diagnostic modeling. Research outcomes expected are: (1) an improved TTCI available through the web and accompanied by interpretive tools for use by faculty; (2) an in-depth, multi-level analysis of the assessment validity and diagnostic capacity of the TTCI; (3) a comprehensive inventory of possible formative and summative uses of CIs, including what is needed by STEM practitioners to make good use of CIs; and (4) a research and development framework for designing, developing, evaluating, improving, and/or implementing CIs in other STEM areas. In general, the study will contribute to the better understanding of student learning and more authentic diagnostic and formative assessment, leading to better learning and teaching of important concepts in STEM. The use of the TTCI will elicit student misconceptions in the thermal and transport engineering sciences such as fluid mechanics, heat transfer, and thermodynamics. The TTCI particularly focuses on concepts deemed important but difficult for undergraduate engineering students. Thus, the proposed research is likely to lead to expanded uses of the TTCI and other CIs for formative classroom purposes as well as for a variety of summative purposes. This project are both an expansion of the psychometric modeling and methods for applying diagnostic models to CIs, and an expanded assessment validity in the area of measurement and assessment of key concepts in science and engineering.