Harvesting Actionable Results for Learning and Instruction: A Novel Mixed Methods Approach to Extracting and Validating Information from Diagnostic Assessment

认知诊断模型(CDM)是心理测量学研究的一个领域，在过去十年中有了实质性的增长。这些评估工具受到了更多的关注，因为简单的总体测试分数不能满足教学目标，因为需要对学生的技能进行更丰富的评估，以支持量身定做的教学。研究者建议扩展现有的CDM心理测量学方法来开发基于高中物理课程的形成性评价设计的诊断层面状态模型(DFSM)。现有的疾病预防控制措施存在一些工作上的问题。它们不能很好地扩展到高维刻面空间，它们需要基于专家输入的复杂编码方案，它们不能探索复杂的刻面关系，或基于选项的名义响应。该项目将建立在现有清洁发展机制方法的基础上，以解决这些限制。该项目开发了一个新的心理测量学模型DFSM和一个纵向扩展，它可以同时模拟高维目标和项目选项水平的中间理解。PI开发了一种机器学习方法来识别各方面之间的关系，产生了一个全面的“方面图”，揭示了属性层次结构和用于物理学习的连接方面。最后，研究人员进行定性研究，以验证DFSM和纵向DFSM的学习和指导输出。该项目得到了NSF的STEM教育研究EDU核心研究能力建设计划(ECR：BCSER)的支持，该计划旨在建设研究人员开展高质量STEM教育研究的能力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Cognitive diagnostic models (CDMs) are an area of psychometric research that has seen substantial growth in the past decade. These assessment tools have received more attention because a simple overall test score does not serve teaching goals as a richer evaluation of the student's skills is needed to support tailored instruction. The investigator proposes to extend existing CDM psychometric approaches to develop the diagnostic facet status model (DFSM) for formative assessment design based on the high school physics curriculum. There are a number of working concerns with existing CDMs. They do not scale well to high-dimensional facet spaces, they require complex coding schemes based on expert input, they cannot explore intricate facet relationships, or option-based nominal responses. This project will build on existing CDM approaches to address these limitations. The project develops a new psychometric model, the DFSM, and a longitudinal extension, which can simultaneously model high-dimensional goal and intermediate understandings at item option level. The PIs develop a machine learning method to identify relations among facets, yielding a comprehensive "facet map" that reveals both attribute hierarchies and conjoined facets for the learning of physics. Finally, the researchers conduct qualitative studies to validate DFSM and longitudinal DFSM's output for learning and instruction.The project is supported by NSF's EDU Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, which is designed to build investigators' capacity to carry out high-quality STEM education research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.