Test bias in a cognitive test: differential item functioning in the CASI
Test bias in a cognitive test: differential item functioning in the CASI
复制标题
DOI：
10.1002/sim.1713
复制
发表时间：
2004-01-30
影响因子：
2
通讯作者：
Larson, EB
中科院分区：
文献类型：
--
作者：
Crane, PK;van Belle, G;Larson, EB
测试偏差的评估是建立测试的结构效度的重要依据。对不同项目功能(DIF)的评估是这一过程中重要的第一步。当来自不同组别的考生在控制了总体能力水平后，在一个项目上有不同的成功概率时，就会出现DIF。在这里，我们提出了在认知评估筛查工具(CASI)中对DIF的分析，使用了一项来自老年人的大型队列研究的数据。我们开发了一种顺序Logistic回归建模技术来评估DIF的测试项目。认知能力的评估是根据对CASI项目的反应通过两种方式获得的：根据原始测试说明使用传统的CASI评分以及使用项目反应理论(IRT)评分。对潜在的DIF的几个人口统计特征进行了检查，包括种族和性别(作为二分变量进入模型)，以及教育年限和年龄(作为连续变量进入)。我们发现，令人失望的是，在这些人口统计学变量中，至少有一个项目有DIF。传统CASI评分的DIF多于IRT评分的DIF。这项研究展示了一种在心理测量测试中评估DIF的强大技术。有如此多的CASI项目有DIF这一发现表明，之前通过CASI测量的群体之间认知功能差异的发现可能是由于测试项目的偏见，而不是群体之间的真正差异。对IRT评分降低DIF影响的发现进行了讨论。对如何处理认知测验中发现的有DIF的项目提出了一些初步建议。我们还讨论了我们开发的DIF检测技术相对于其他评估DIF的技术的优势。版权所有(C)2004 John Wiley Sons，Ltd.
Assessment of test bias is important to establish the construct validity of tests. Assessment of differential item functioning (DIF) is an important first step in this process. DIF is present when examinees from different groups have differing probabilities of success on an item, after controlling for overall ability level. Here, we present analysis of DIF in the Cognitive Assessment Screening Instrument (CASI) using data from a large cohort study of elderly adults. We developed an ordinal logistic regression modelling technique to assess test items for DIF. Estimates of cognitive ability were obtained in two ways based on responses to CASI items: using traditional CASI scoring according to the original test instructions as well as using item response theory (IRT) scoring. Several demographic characteristics were examined for potential DIF, including ethnicity and gender (entered into the model as dichotomous variables), and years of education and age (entered as continuous variables). We found that a disappointingly large number of items had DIF with respect to at least one of these demographic variables. More items were found to have DIF with traditional CASI scoring than with IRT scoring. This study demonstrates a powerful technique for the evaluation of DIF in psychometric tests. The finding that so many CASI items had DIF suggests that previous findings of differences between groups in cognitive functioning as measured by the CASI may be due to biased test items rather than true differences between groups. The finding that IRT scoring diminished the impact of DIF is discussed. Some preliminary suggestions for how to deal with items found to have DIF in cognitive tests are made. The advantages of the DIF detection techniques we developed are discussed in relation to other techniques for the evaluation of DIF. Copyright (C) 2004 John Wiley Sons, Ltd.