Formative Assessment in Mathematics: Current Status and Guidelines for Future Developments

为了应对形成性评估实践在加州的PK-12数学教室中的使用越来越多，加州大学戴维斯分校和克莱蒙研究生院共同努力，调查形成性评估正在使用什么，从业者和学生如何利用这些评估，以及它们如何影响总结性状态评估的表现。本研究的目的是探讨以下四个问题：（1）形成性评价在加州中小学的使用现状。(2)当前形成性评估的内容和心理测量学特征是什么？ (3)形成性评估的结果对学生的学业成就有什么影响？以及（4）形成性评价的使用是否对学生的亚群体，如英语学习者、有色人种学生、经济困难学生和残疾学生的表现产生不同的影响？本研究分三个阶段，采用混合方法回答四个研究问题。为了回答研究问题1，第一阶段对该州三个主要地区（北方、中部和南部）的所有学校校长进行在线调查。对数据进行频率和描述性分析，以准确了解形成性评估的现状。软件用于定性分析对开放式问题的答复。不同背景的受访者和不同特点的学校之间的分类反应和李克特型反应的平均值和标准差的频率进行了比较。为了解决研究问题2，第二阶段进行了一系列的心理测量和内容分析现有的评估，以确定有效性，可靠性和评估的可访问性。共有120所PK-12学校参加了这些分析，这些学校拥有最强的形成性评估工具和对总结性评估影响最大的形成性评估。该阶段的样本量代表4级跨度（PK K、4级、8级和10级），通过功效分析确定，最小可检测差异为0.25标准差，统计功效为80%，I类错误为0.05。由于研究的重点是传统上表现不佳和代表性不足的人群，分层随机抽样的方法，其中亚组标识符作为分层变量。为了评估内容属性，该研究利用了Webb（1997）的对齐方法。形成性评估的可靠性采用内部一致性方法（Cronbach alpha）进行检验。有效性分析通过内容和标准相关的有效性方法，包括多特征/多方法的方法。为了回答研究问题3和4，创建了一系列路径模型，以探索形成性评估在预测学生在国家年终评估中的表现的力量。此外，进行了事后因素设计，以调查是否一个或多个预先存在的条件可能导致随后的差异，在组的参与者。为了记录成功的做法，研究的第三阶段对六所学校进行了案例研究，这些学校采用了有效的形成性评估做法，并使用总结性措施证明了学术成长。个案研究包括与教师和管理人员的焦点小组访谈录音，教师会议的录像观察，以及课堂观察。这项研究的主要成果将是一套研究知情和实地测试的结论，建议，指导方针和工具，用于开发和使用新的或改进的PK-12数学形成性评估。结果工具包括一个蓝图和测试规格的基础上加州国家数学内容标准的8年级，和一个实地测试的形成性评估原型，重点是关键的代数概念和技能的年级水平。加州大学洛杉矶分校评价研究中心担任项目外部评价员。

In response to the increased use of formative assessment practices in California's PK-12 mathematics classrooms, the University of California-Davis and Claremont Graduate University join efforts to investigate what formative assessments are in use, how practitioners and students are utilizing these assessments, and how they impact performance on summative state assessments. Four research questions guide this study: (1) What is the current status of formative assessment use in California's PK & K, elementary, middle, and high schools? (2) What are the content and psychometric properties of the current formative assessments? (3) What impact do formative assessment outcomes have on student academic achievement as measured by summative assessments? and (4) Does the use of formative assessment differentially impact performance of subgroups of students, such as ELLs, students of color, economically disadvantaged students, and students with disabilities? The study consists of three phases and employs a mixed-methods approach to answer the four research questions. To answer research question 1, Phase I administers an online survey to all school principals in three major regions of the State (northern, central, and southern). Frequency and descriptive analyses of data are conducted to obtain an accurate view of the current status of formative assessments. Software is used to qualitatively analyze responses to open-ended questions. Frequencies of categorical responses and mean and standard deviations of Likert-type responses are compared across respondents with different backgrounds and schools with different characteristics. To address research question 2, Phase II conducts a series of psychometric and content analyses of existing assessments to ascertain validity, reliability, and accessibility of assessments. A total of 120 PK-12 schools with the strongest formative assessment instruments and the greatest formative assessment impact on summative assessments participate in these analyses. Sample sizes for this phase, representative of a four-grade span (PK & K, 4th, 8th, and 10th grades), were determined through a power analysis for a minimum of 0.25 standard deviation detectable difference with a statistical power of 80% at a 0.05 type I error. Due to the study's focus on traditionally low-performing and underrepresented populations, a stratified random sampling approach in which subgroup identifiers are used as stratification variables is employed. To assess content properties, the study utilizes Webb's (1997) alignment methodology. Reliability of formative assessments is examined using the internal consistency approach (Cronbach alpha). Validity is analyzed through content- and criterion-related validity approaches, including a Multi-Traits/Multi-Methods approach. To answer research questions 3 and 4, a series of path models are created to explore the power of formative assessments in predicting student performance on the State's end-of-year assessments. In addition, an Ex Post Facto design is conducted to investigate whether one or more preexisting conditions have possibly caused subsequent differences in the groups of participants. To document successful practices, the study's Phase III develops case studies of six schools with effective formative assessment practices and demonstrated academic growth using summative measures. Case studies include audiotaped focus group interviews with teachers and administrators, videotaped observations of teacher meetings, and classroom observations. The main outcome of this study will be a set of research-informed and field-tested conclusions, recommendations, guidelines, and tools for the development and use of new or improved PK-12 mathematics formative assessments. Outcome tools include a blueprint and test specifications based on the California State Mathematics Content Standards for 8th grade, and a field-tested formative assessment prototype focused on key algebra concepts and skills for that grade level. The Center for the Study of Evaluation at the University of California-Los Angeles serves as the project external evaluator.