A Family of Diagnostic Models for Evaluating Learning Progressions

本研究计划将发展心理测量方法，以实证评估发展进程。发展进程描述了个人或群体认知、心理或行为发展的理论或观察顺序。在教育环境中，学习进阶描述了学生随着时间学习特定内容领域而发展的越来越复杂的推理方式。尽管他们的流行和效用，定量方法的发展，以评估学习进度已经停滞不前。该项目将通过为发展过程建模提供一个现代的、多维的、纵向的框架，推动心理测量学和学习科学领域的发展。虽然该项目侧重于教育应用，但开发的方法将广泛适用于社会和行为科学的学科。该项目将从一个代表性不足的群体中培养一名研究生。将开发免费和易于使用的软件，供研究人员在自己的学习进度考试中使用。该项目的结果和产品有可能改变研究人员在发展进程的实证评估中设计、解释和分析评估的方式。研究者将使用诊断分类模型（DCM）框架来模拟学习进程。dcm是一种多变量心理测量模型，它将考生划分为特定的类别潜在特征水平（例如，基本、熟练、高级）。dcm在教育环境中变得很有吸引力，因为它们以分类的形式提供了非常需要的诊断和标准参考分数解释。最近，dcm在纵向背景下得到了发展，为学生成长提供了标准参考解释。为了对学习过程进行建模，所开发的模型将结合广义纵向DCM和用于建模属性层次结构的分层DCM。建模框架的这种融合允许同时检查属性层次和学生随着时间的学习，它们一起构成了学习进程的基础。模拟研究将指导和告知开发方法的实际应用，涉及数据要求（即项目数量或样本量），测试设计，模型拟合以及影响基于模型的推断的准确性，有效性和可靠性的因素。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will develop psychometric methodology to empirically evaluate developmental progressions. A developmental progression describes a theorized or observed sequence of cognitive, psychological, or behavioral developments in an individual or group. In educational settings, learning progressions describe the increasingly sophisticated ways of reasoning that develop as students learn about specific content domains over time. Despite their prevalence and utility, quantitative methodological developments to evaluate learning progressions have stagnated. This project will advance the fields of psychometrics and learning sciences by providing a modern, multidimensional, and longitudinal framework for modeling developmental progressions. Although the project focuses on educational applications, the developed methods will be widely applicable in disciplines across the social and behavioral sciences. The project will train a graduate student from an underrepresented group. Free and easy-to-use software will be developed for researchers to utilize in their own examinations of learning progressions. The results and products stemming from this project have the potential to change the way researchers design, interpret, and analyze assessments in the empirical evaluation of developmental progressions.The investigator will use a diagnostic classification model (DCM) framework to model learning progressions. DCMs are multivariate psychometric models that classify examinees into specified levels of categorical latent traits (e.g., basic, proficient, advanced). DCMs have become attractive in educational settings because they provide much desired diagnostic and criterion-referenced score interpretations in the form of classifications. Recently, DCMs have been developed for longitudinal contexts that provide criterion-referenced interpretations of student growth. To model learning progressions, the developed model will combine a generalized longitudinal DCM with the hierarchical DCM designed to model attribute hierarchies. This fusion of modeling frameworks allows for the simultaneous examination of attribute hierarchies and student learning over time, which together comprise the basis of a learning progression. Simulation studies will guide and inform the practical application of the developed methods with respect to data requirements (i.e., number of items or sample size), test design, model fit, and factors impacting the accuracy, validity, and reliability of model-based inferences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.