Monitoring Scale Scores over Time via Quality Control Charts, Model-Based Approaches, and Time Series Techniques
Monitoring Scale Scores over Time via Quality Control Charts, Model-Based Approaches, and Time Series Techniques
复制标题
通过质量控制图、基于模型的方法和时间序列技术随时间监控量表得分
DOI：
10.1007/s11336-013-9317-5
复制
发表时间：
2013
期刊：
影响因子：
3
通讯作者：
A. A. Davier
中科院分区：
文献类型：
--
作者：
Yi;A. A. Davier
随着时间的推移，保持一个稳定的分数尺度是所有标准化教育评估的关键。用于评估量表漂移的传统质量控制工具和方法要么需要特殊的等值设计，要么可能过于耗时，无法定期考虑使用在管理和其分数报告之间具有短时间窗口的操作测试。因此，传统的方法不足以及时捕捉异常的测试结果。本文提出了一种新的方法来监测和评估量表的漂移。它涉及质量控制图，基于模型的方法，和时间序列技术，以适应以下需要的监测规模的分数：连续监测，调整习惯的变化，识别突变，自相关的评估。使用操纵数据的基础上的真实的反应，从71个大规模的高风险的语言评估的方法的性能进行评估。
Maintaining a stable score scale over time is critical for all standardized educational assessments. Traditional quality control tools and approaches for assessing scale drift either require special equating designs, or may be too time-consuming to be considered on a regular basis with an operational test that has a short time window between an administration and its score reporting. Thus, the traditional methods are not sufficient to catch unusual testing outcomes in a timely manner. This paper presents a new approach for score monitoring and assessment of scale drift. It involves quality control charts, model-based approaches, and time series techniques to accommodate the following needs of monitoring scale scores: continuous monitoring, adjustment of customary variations, identification of abrupt shifts, and assessment of autocorrelation. Performance of the methodologies is evaluated using manipulated data based on real responses from 71 administrations of a large-scale high-stakes language assessment.