Using Rule Space and Poset-based Adaptive Testing Methodologies to Identify Ability Patterns in Early Mathematics and Create a Comprehensive Mathematics Ability Test

一项针对3-7岁儿童的新评估正在开发中，旨在为教师提供关于儿童数学能力发展的诊断信息，涉及10个领域，如计数、排序、加减法和测量。基于研究的综合数学评估（CREMA）正在使用创新的心理测量模型来揭示儿童在10个领域中每个领域的特定属性的信息。CREMA将使用计算机自适应测试，在相对较短的时间内根据精心开发的学习轨迹产生信息。该项目以两个目标为指导：1)产生一种认知诊断适应性评估，该评估将产生比以前更有用和详细的关于学生数学知识的信息；2)使用尖端的心理测量方法对发展过程进行密切的认知诊断。正在开发一个大约350个项目的项目库，可用于确定3-7岁儿童对10个领域的理解水平，这些领域已被确定为进一步学习数学的基础。由布法罗大学道格拉斯·克莱门茨博士领导的研究小组在凯斯西储大学统计学家柯蒂斯·达冈博士的协助下进行了开发工作。CREMA正在使用基于q矩阵理论、规则状态模型和假设集的前沿心理测量模型进行开发。最初的项目池包括来自REMA的项目，REMA是以前开发的基于单维IRT模型的工具。新项目将在800名学生中至少200名学生中进行试点，这些学生平均分布在学前班到二年级。然后使用成功的项目创建新的CREMA。新的评估正在对300名学前班到二年级的孩子进行实地测试。随机抽取50名学生（每个年级至少10名），在他们做题时进行录像。特定的收敛标准被用于反馈特定项目如何执行以满足所需的规范。外部评估人员正在审核流程，并对项目编码和其他分析进行抽查。主要产品将是将广泛提供的CREMA。这种使用计算机自适应测试的仪器将为教师提供有关幼儿对关键数学概念理解的现成信息。新的心理测量模型将用于处理来自单个项目的多个属性，这将为幼儿数学评估领域的发展做出巨大的努力。一家出版商表示有兴趣广泛提供评估，以增加评估对幼儿数学学习产生重大影响的可能性。

A new assessment for children ages 3-7 is being developed to provide teachers with diagnostic information on a child's development of mathematics facility on ten domains such as counting, sequencing, adding/subtracting, and measurement. The Comprehensive Research-based Mathematics Assessment (CREMA) is being developed using innovative psychometric models to reveal information about children on specific attributes for each of the 10 domains. The CREMA will produce information based on carefully developed learning trajectories in a relative short period of time by using computer adaptive testing. The project is guided by two goals: 1) to produce a cognitively diagnostic adaptive assessment that will yield more useful and detailed information about students' knowledge of mathematics than previously possible, and 2) subject the developmental progressions to close cognitive diagnosis using cutting-edge psychometric approaches. An item pool of about 350 items is being developed that can be used to identify the level of understanding children ages 3-7 have on the 10 domains that have been identified as foundational to further learning in mathematics. A research team headed by Dr. Douglas Clements at the University of Buffalo is conducting the development work while being assisted by Dr. Curtis Tatsuoka, a statistician at Case Western Reserve University.The CREMA is being developed using leading-edge psychometric models based on Q-Matrix theory, rule-state models, and posets. The initial item pool includes items from the REMA, a previously developed instrument based on unidemensional IRT models. New items are being piloted with at least 200 students from a group of a total of 800 students evenly distributed among pre-K to grade 2. The successful items then are used to create the new CREMA. The new assessment is being field tested with 300 children, pre-K to grade 2. A random sample of 50 students (at least 10 from each grade) is being video taped as they work the items. Specific criteria of convergence are being used for feedback on how specific items are performing to meet the required specifications. An external evaluator is auditing the process and is doing spot checks of item codings and other analyses performed.The main product will be the CREMA that will be made widely available. This instrument using computer adaptive testing will provide teachers with ready information on young children's understanding of critical mathematical ideas. The new psychometric models that will be used and developed to process multiple attributes from individual items will make large strives to move forward the field of mathematics assessment of young children. A publisher has expressed interest to make the assessment widely available that increases the likelihood the assessment will have large impact on early childhood mathematics learning.