Formative Assessment in Mathematics: Current Status and Guidelines for Future Developments

为了应对加州PK-12数学课堂上越来越多地使用形成性评估实践，加州大学戴维斯分校和克莱蒙特研究生院共同努力，调查使用了什么形成性评估，实践者和学生如何利用这些评估，以及它们如何影响终结性状态评估的表现。本研究以四个研究问题为指导：(1)加州小学、初中和高中形成性评估的使用现状如何？(2)当前形成性评估的内容和心理测量学特性是什么？(3)形成性评估结果对终结性评估衡量的学生学业成绩有什么影响？以及(4)形成性评估的使用是否对学生亚群的表现产生了不同的影响，如学生、有色人种学生、经济困难学生和残疾学生？研究分为三个阶段，采用混合方法回答四个研究问题。为了回答研究问题1，第一阶段对该州三个主要地区(北部、中部和南部)的所有校长进行了在线调查。对数据进行频率和描述性分析，以准确了解形成性评估的现状。软件用于定性分析对开放式问题的回答。本研究比较了不同背景、不同学校的被调查者的分类回答频率以及李克特类型回答的均值和标准差。为了解决研究问题2，第二阶段对现有评估进行了一系列心理测量和内容分析，以确定评估的有效性、可靠性和可及性。共有120所形成性评估工具最强、形成性评估对终结性评估影响最大的PK-12学校参与了这些分析。这一阶段的样本大小代表四个等级的跨度(PK&Amp；K、4、8和10个等级)，通过功率分析确定了最小0.25标准偏差的可检测差异，在0.05类型I误差下的统计功率为80%。由于这项研究的重点是传统上表现不佳和代表性不足的群体，因此采用了一种分层随机抽样方法，其中亚组标识符用作为分层变量。为了评估内容属性，该研究使用了Webb(1997)的对齐方法。使用内部一致性方法(Cronbach Alpha)检查形成性评估的可靠性。效度分析通过内容和标准相关的效度方法，包括多特质/多方法方法。为了回答研究问题3和4，创建了一系列路径模型，以探索形成性评估在预测学生在国家年终评估中的表现方面的力量。此外，还进行了事后设计，以调查一种或多种先前存在的疾病是否可能导致参与者群体随后的差异。为了记录成功的做法，研究的第三阶段开发了六所学校的案例研究，这些学校采用了有效的形成性评估做法，并使用终结性措施展示了学业成长。案例研究包括对教师和管理人员进行焦点小组访谈的录音、教师会议的录像观察和课堂观察。这项研究的主要结果将是一套经过研究和实地测试的结论、建议、指南和工具，用于开发和使用新的或改进的PK-12数学形成性评估。结果工具包括一份蓝图和基于加利福尼亚州数学内容标准的八年级测试规范，以及一个经过实地测试的形成性评估原型，重点是该年级水平的关键代数概念和技能。加州大学洛杉矶分校的评估研究中心是项目的外部评估者。

In response to the increased use of formative assessment practices in California's PK-12 mathematics classrooms, the University of California-Davis and Claremont Graduate University join efforts to investigate what formative assessments are in use, how practitioners and students are utilizing these assessments, and how they impact performance on summative state assessments. Four research questions guide this study: (1) What is the current status of formative assessment use in California's PK & K, elementary, middle, and high schools? (2) What are the content and psychometric properties of the current formative assessments? (3) What impact do formative assessment outcomes have on student academic achievement as measured by summative assessments? and (4) Does the use of formative assessment differentially impact performance of subgroups of students, such as ELLs, students of color, economically disadvantaged students, and students with disabilities? The study consists of three phases and employs a mixed-methods approach to answer the four research questions. To answer research question 1, Phase I administers an online survey to all school principals in three major regions of the State (northern, central, and southern). Frequency and descriptive analyses of data are conducted to obtain an accurate view of the current status of formative assessments. Software is used to qualitatively analyze responses to open-ended questions. Frequencies of categorical responses and mean and standard deviations of Likert-type responses are compared across respondents with different backgrounds and schools with different characteristics. To address research question 2, Phase II conducts a series of psychometric and content analyses of existing assessments to ascertain validity, reliability, and accessibility of assessments. A total of 120 PK-12 schools with the strongest formative assessment instruments and the greatest formative assessment impact on summative assessments participate in these analyses. Sample sizes for this phase, representative of a four-grade span (PK & K, 4th, 8th, and 10th grades), were determined through a power analysis for a minimum of 0.25 standard deviation detectable difference with a statistical power of 80% at a 0.05 type I error. Due to the study's focus on traditionally low-performing and underrepresented populations, a stratified random sampling approach in which subgroup identifiers are used as stratification variables is employed. To assess content properties, the study utilizes Webb's (1997) alignment methodology. Reliability of formative assessments is examined using the internal consistency approach (Cronbach alpha). Validity is analyzed through content- and criterion-related validity approaches, including a Multi-Traits/Multi-Methods approach. To answer research questions 3 and 4, a series of path models are created to explore the power of formative assessments in predicting student performance on the State's end-of-year assessments. In addition, an Ex Post Facto design is conducted to investigate whether one or more preexisting conditions have possibly caused subsequent differences in the groups of participants. To document successful practices, the study's Phase III develops case studies of six schools with effective formative assessment practices and demonstrated academic growth using summative measures. Case studies include audiotaped focus group interviews with teachers and administrators, videotaped observations of teacher meetings, and classroom observations. The main outcome of this study will be a set of research-informed and field-tested conclusions, recommendations, guidelines, and tools for the development and use of new or improved PK-12 mathematics formative assessments. Outcome tools include a blueprint and test specifications based on the California State Mathematics Content Standards for 8th grade, and a field-tested formative assessment prototype focused on key algebra concepts and skills for that grade level. The Center for the Study of Evaluation at the University of California-Los Angeles serves as the project external evaluator.