A Family of Diagnostic Models for Evaluating Learning Progressions

这项研究项目将开发心理测量学方法来经验地评估发展进程。发展进程描述了一个人或群体的认知、心理或行为发展的理论上或观察到的序列。在教育环境中，学习进度描述了随着时间的推移，随着学生学习特定内容领域而发展起来的日益复杂的推理方式。尽管它们普遍存在和有用，但评估学习进展的量化方法发展却停滞不前。这个项目将通过提供一个现代的、多维的和纵向的框架来模拟发展进程，从而推动心理测量学和学习科学领域的发展。尽管该项目侧重于教育应用，但开发的方法将广泛应用于社会科学和行为科学的各个学科。该项目将从代表性不足的群体中培养一名研究生。将开发免费和易于使用的软件，供研究人员在他们自己的学习进度考试中使用。这个项目的结果和产品有可能改变研究人员设计、解释和分析发展进步实证评估中的评估的方式。研究人员将使用诊断分类模型(DCM)框架来模拟学习进展。DCMS是一种多变量心理测量学模型，它将考生分类为特定水平的类别潜在特质(例如，基本、熟练、高级)。DCMS在教育环境中变得很有吸引力，因为它们以分类的形式提供了非常理想的诊断和标准参考分数解释。最近，DCMS已经被开发用于提供对学生成长的标准参考解释的纵向背景。为了模拟学习过程，开发的模型将结合广义的纵向DCM和设计用于模拟属性层次的层次化DCM。这种建模框架的融合允许同时检查属性层次结构和学生随着时间的推移而进行的学习，它们共同构成了学习进程的基础。模拟研究将指导和告知开发的方法在数据要求(即项目数或样本量)、测试设计、模型拟合以及影响基于模型的推理的准确性、有效性和可靠性的因素方面的实际应用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will develop psychometric methodology to empirically evaluate developmental progressions. A developmental progression describes a theorized or observed sequence of cognitive, psychological, or behavioral developments in an individual or group. In educational settings, learning progressions describe the increasingly sophisticated ways of reasoning that develop as students learn about specific content domains over time. Despite their prevalence and utility, quantitative methodological developments to evaluate learning progressions have stagnated. This project will advance the fields of psychometrics and learning sciences by providing a modern, multidimensional, and longitudinal framework for modeling developmental progressions. Although the project focuses on educational applications, the developed methods will be widely applicable in disciplines across the social and behavioral sciences. The project will train a graduate student from an underrepresented group. Free and easy-to-use software will be developed for researchers to utilize in their own examinations of learning progressions. The results and products stemming from this project have the potential to change the way researchers design, interpret, and analyze assessments in the empirical evaluation of developmental progressions.The investigator will use a diagnostic classification model (DCM) framework to model learning progressions. DCMs are multivariate psychometric models that classify examinees into specified levels of categorical latent traits (e.g., basic, proficient, advanced). DCMs have become attractive in educational settings because they provide much desired diagnostic and criterion-referenced score interpretations in the form of classifications. Recently, DCMs have been developed for longitudinal contexts that provide criterion-referenced interpretations of student growth. To model learning progressions, the developed model will combine a generalized longitudinal DCM with the hierarchical DCM designed to model attribute hierarchies. This fusion of modeling frameworks allows for the simultaneous examination of attribute hierarchies and student learning over time, which together comprise the basis of a learning progression. Simulation studies will guide and inform the practical application of the developed methods with respect to data requirements (i.e., number of items or sample size), test design, model fit, and factors impacting the accuracy, validity, and reliability of model-based inferences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.