Formative Assessment in Mathematics: Current Status and Guidelines for Future Developments

为了应对加州PK-12数学课堂中形成性评估实践的增加使用，加州大学戴维斯分校和克莱蒙特研究生大学共同努力调查正在使用的形成性评估，从业者和学生如何利用这些评估，以及它们如何影响总结性州评估的表现。四个研究问题指导了本研究：(1)加利福尼亚州PK & K，小学，初中和高中的形成性评估使用现状如何？(2)当前形成性评估的内容和心理特征是什么？(3)形成性评估结果对总结性评估衡量的学生学业成绩有什么影响？(4)形成性评估的使用是否会对不同学生群体的表现产生不同的影响，如ELLs、有色人种学生、经济困难学生和残疾学生？该研究包括三个阶段，并采用混合方法的方法来回答四个研究问题。为了回答研究问题1，第一阶段对该州三个主要地区（北部、中部和南部）的所有学校校长进行了在线调查。对数据进行频率和描述性分析，以便准确了解形成性评价的现状。软件用于定性分析对开放式问题的回答。比较了不同背景和不同学校的被调查者的分类反应频率、likert型反应的均值和标准差。为了解决研究问题2，第二阶段对现有的评估进行了一系列的心理测量和内容分析，以确定评估的有效性、可靠性和可及性。共有120所具有最强形成性评估工具和最大形成性评估对总结性评估影响的PK-12学校参与了这些分析。该阶段的样本量代表了四个年级（PK & K、4年级、8年级和10年级），通过功率分析确定了最小为0.25标准差的可检测差异，统计功率为80%，类型1误差为0.05。由于该研究的重点是传统上表现不佳和代表性不足的人群，因此采用分层随机抽样方法，其中使用子组标识符作为分层变量。为了评估内容属性，本研究采用了Webb（1997）的对齐方法。形成性评估的可靠性使用内部一致性方法（Cronbach alpha）进行检验。效度分析采用与内容和效度相关的方法，包括多特征/多方法方法。为了回答研究问题3和4，我们创建了一系列路径模型，以探索形成性评估在预测学生在国家年终评估中的表现方面的力量。此外，一个事后的设计进行调查是否一个或多个先前存在的条件可能导致了参与者群体的后续差异。为了记录成功的实践，该研究的第三阶段对六所学校进行了案例研究，这些学校采用了有效的形成性评估实践，并使用总结性措施展示了学术增长。案例研究包括对教师和行政人员的焦点小组访谈录音、对教师会议的观察录像和课堂观察。这项研究的主要成果将是一套研究信息和实地测试的结论、建议、指导方针和工具，用于开发和使用新的或改进的PK-12数学形成性评估。成果工具包括基于加州八年级数学内容标准的蓝图和测试规范，以及侧重于该年级关键代数概念和技能的实地测试形成性评估原型。加州大学洛杉矶分校评估研究中心担任该项目的外部评估机构。

In response to the increased use of formative assessment practices in California's PK-12 mathematics classrooms, the University of California-Davis and Claremont Graduate University join efforts to investigate what formative assessments are in use, how practitioners and students are utilizing these assessments, and how they impact performance on summative state assessments. Four research questions guide this study: (1) What is the current status of formative assessment use in California's PK & K, elementary, middle, and high schools? (2) What are the content and psychometric properties of the current formative assessments? (3) What impact do formative assessment outcomes have on student academic achievement as measured by summative assessments? and (4) Does the use of formative assessment differentially impact performance of subgroups of students, such as ELLs, students of color, economically disadvantaged students, and students with disabilities? The study consists of three phases and employs a mixed-methods approach to answer the four research questions. To answer research question 1, Phase I administers an online survey to all school principals in three major regions of the State (northern, central, and southern). Frequency and descriptive analyses of data are conducted to obtain an accurate view of the current status of formative assessments. Software is used to qualitatively analyze responses to open-ended questions. Frequencies of categorical responses and mean and standard deviations of Likert-type responses are compared across respondents with different backgrounds and schools with different characteristics. To address research question 2, Phase II conducts a series of psychometric and content analyses of existing assessments to ascertain validity, reliability, and accessibility of assessments. A total of 120 PK-12 schools with the strongest formative assessment instruments and the greatest formative assessment impact on summative assessments participate in these analyses. Sample sizes for this phase, representative of a four-grade span (PK & K, 4th, 8th, and 10th grades), were determined through a power analysis for a minimum of 0.25 standard deviation detectable difference with a statistical power of 80% at a 0.05 type I error. Due to the study's focus on traditionally low-performing and underrepresented populations, a stratified random sampling approach in which subgroup identifiers are used as stratification variables is employed. To assess content properties, the study utilizes Webb's (1997) alignment methodology. Reliability of formative assessments is examined using the internal consistency approach (Cronbach alpha). Validity is analyzed through content- and criterion-related validity approaches, including a Multi-Traits/Multi-Methods approach. To answer research questions 3 and 4, a series of path models are created to explore the power of formative assessments in predicting student performance on the State's end-of-year assessments. In addition, an Ex Post Facto design is conducted to investigate whether one or more preexisting conditions have possibly caused subsequent differences in the groups of participants. To document successful practices, the study's Phase III develops case studies of six schools with effective formative assessment practices and demonstrated academic growth using summative measures. Case studies include audiotaped focus group interviews with teachers and administrators, videotaped observations of teacher meetings, and classroom observations. The main outcome of this study will be a set of research-informed and field-tested conclusions, recommendations, guidelines, and tools for the development and use of new or improved PK-12 mathematics formative assessments. Outcome tools include a blueprint and test specifications based on the California State Mathematics Content Standards for 8th grade, and a field-tested formative assessment prototype focused on key algebra concepts and skills for that grade level. The Center for the Study of Evaluation at the University of California-Los Angeles serves as the project external evaluator.