Comparing Methods for Assessing Reliability.
Comparing Methods for Assessing Reliability.
复制标题
评估可靠性的方法比较。
DOI：
10.1093/jssam/smaa018
复制
发表时间：
2021
影响因子：
2.1
通讯作者：
Yan,Ting
中科院分区：
文献类型：
--
作者：
Tourangeau,Roger;Sun,Hanyu;Yan,Ting
评估调查数据可靠性的通常方法是，在初次访谈后的短时间内（如一至两周）进行重新访谈，并利用这些数据估计相对简单的统计数据，如总差异率。更复杂的方法也被用来估计可靠性。这些包括估计多性状，多方法实验，模型应用于纵向数据，和潜在的类分析。据我们所知，没有以前的研究系统地比较了这些不同的方法来评估可靠性。烟草与健康可靠性和有效性的人口评估（PATH-RV）研究，在一个国家的概率样本进行，评估了从PATH研究的第4波问卷的答案的可靠性。PATH-RV中的受访者接受了两次约两周的采访。我们研究了经典的调查方法是否会得出与更复杂的方法不同的结论。我们还研究了两个前antemethods评估问题的调查问题和项目的无应答率和响应时间，看看这些有多强的相关性不同的可靠性估计。我们发现，Kappa与GDR和随时间推移的相关性高度相关，但后两个统计数据的相关性较低，特别是对于成年受访者;主要PATH研究中相同项目的纵向分析估计值也与传统的可靠性估计值高度相关。基于较少项目的潜在类别分析结果也与传统措施表现出高度一致。其他方法和指标充其量与来自重新访谈数据的可靠性估计的关系较弱。虽然问题理解援助似乎挖掘一个不同的因素，从其他措施，对于成年受访者，它没有预测项目无反应和反应迟缓，因此可能是一个有用的辅助传统的措施。
The usual method for assessing the reliability of survey data has been to conduct reinterviews a short interval (such as one to two weeks) after an initial interview and to use these data to estimate relatively simple statistics, such as gross difference rates (GDRs). More sophisticated approaches have also been used to estimate reliability. These include estimates from multi-trait, multi-method experiments, models applied to longitudinal data, and latent class analyses. To our knowledge, no prior study has systematically compared these different methods for assessing reliability. The Population Assessment of Tobacco and Health Reliability and Validity (PATH-RV) Study, done on a national probability sample, assessed the reliability of answers to the Wave 4 questionnaire from the PATH Study. Respondents in the PATH-RV were interviewed twice about two weeks apart. We examined whether the classic survey approach yielded different conclusions from the more sophisticated methods. We also examined twoex antemethods for assessing problems with survey questions and item nonresponse rates and response times to see how strongly these related to the different reliability estimates. We found that kappa was highly correlated with both GDRs and over-time correlations, but the latter two statistics were less highly correlated, particularly for adult respondents; estimates from longitudinal analyses of the same items in the main PATH study were also highly correlated with the traditional reliability estimates. The latent class analysis results, based on fewer items, also showed a high level of agreement with the traditional measures. The other methods and indicators had at best weak relationships with the reliability estimates derived from the reinterview data. Although the Question Understanding Aid seems to tap a different factor from the other measures, for adult respondents, it did predict item nonresponse and response latencies and thus may be a useful adjunct to the traditional measures.