Taking the Missing Propensity Into Account When Estimating Competence Scores: Evaluation of Item Response Theory Models for Nonignorable Omissions
Taking the Missing Propensity Into Account When Estimating Competence Scores: Evaluation of Item Response Theory Models for Nonignorable Omissions
复制标题
DOI：
10.1177/0013164414561785
复制
发表时间：
2015-10-01
影响因子：
2.7
通讯作者：
Carstensen, Claus H.
中科院分区：
文献类型：
--
作者：
Koehler, Carmen;Pohl, Steffi;Carstensen, Claus H.
当进行能力测试时，受试者经常省略项目。这些缺失的回答对正确估计熟练程度构成了威胁。较新的基于模型的方法旨在通过将潜在的丢失倾向结合到测量模型中来考虑不可忽略的丢失数据过程。在使用这些模型时，通常会做两个假设：(1)遗漏倾向是一维的；(2)遗漏倾向和能力是二维正态分布的。然而，这些假设在真实数据集中可能会被违反，因此可能会对这种方法的有效性构成威胁。本研究使用来自国家教育小组研究的学校样本(N=15,396)和成人样本(N=7,256)的数据，重点研究不同领域的建模能力。我们的兴趣是调查违反单维性和正态分布假设是否会严重影响基于模型的方法在能力估计差异方面的表现。我们提出了一个具有能力维度、一维缺失倾向和比多元正态分布更灵活的分布假设的模型。与忽略缺失响应的模型相比，使用该模型进行能力估计会产生不同的能力估计。讨论了在大规模评估中对能力评估的影响。
When competence tests are administered, subjects frequently omit items. These missing responses pose a threat to correctly estimating the proficiency level. Newer model-based approaches aim to take nonignorable missing data processes into account by incorporating a latent missing propensity into the measurement model. Two assumptions are typically made when using these models: (1) The missing propensity is unidimensional and (2) the missing propensity and the ability are bivariate normally distributed. These assumptions may, however, be violated in real data sets and could, thus, pose a threat to the validity of this approach. The present study focuses on modeling competencies in various domains, using data from a school sample (N = 15,396) and an adult sample (N = 7,256) from the National Educational Panel Study. Our interest was to investigate whether violations of unidimensionality and the normal distribution assumption severely affect the performance of the model-based approach in terms of differences in ability estimates. We propose a model with a competence dimension, a unidimensional missing propensity and a distributional assumption more flexible than a multivariate normal. Using this model for ability estimation results in different ability estimates compared with a model ignoring missing responses. Implications for ability estimation in large-scale assessments are discussed.