What is the validity evidence for assessments of clinical teaching?
What is the validity evidence for assessments of clinical teaching?
复制标题
DOI：
10.1111/j.1525-1497.2005.0258.x
复制
发表时间：
2005-12-01
影响因子：
5.7
通讯作者：
Mandrekar, JN
中科院分区：
文献类型：
--
作者：
Beckman, TJ;Cook, DA;Mandrekar, JN
背景：虽然在评价评估工具时应使用各种有效性证据，但对教学评估的回顾表明，作者追求的有效性证据范围有限。目的：建立一种评估有效性证据的方法，并将现有临床教学评估工具中支持分数的证据量化。设计：综合检索出22篇关于临床教学评估的文章。使用美国心理学和教育研究协会概述的标准，我们开发了一种方法来对每篇文章中报告的5类有效性证据进行评级。然后，我们通过对每个类别的评分求和来量化有效性证据。我们还计算了加权kappa系数来确定每一类有效性证据的评分者之间的可靠性。MAIN结果：内容和内部结构证据得到了最高的评分(分别为27和32，满分为44)。与其他变量、后果和应对过程的关系得分最低(分别为9分、2分和2分)。内容、内部结构以及与其他变量的关系(kappa范围为0.52至0.96，均为P值)，评分者之间的信度较好
BACKGROUND: Although a variety of validity evidence should be utilized when evaluating assessment tools, a review of teaching assessments suggested that authors pursue a limited range of validity evidence.OBJECTIVES: To develop a method for rating validity evidence and to quantify the evidence supporting scores from existing clinical teaching assessment instruments.DESIGN: A comprehensive search yielded 22 articles on clinical teaching assessments. Using standards outlined by the American Psychological and Education Research Associations, we developed a method for rating the 5 categories of validity evidence reported in each article. We then quantified the validity evidence by summing the ratings for each category. We also calculated weighted kappa coefficients to determine interrater reliabilities for each category of validity evidence.MAIN RESULTS: Content and Internal Structure evidence received the highest ratings (27 and 32, respectively, of 44 possible). Relation to Other Variables, Consequences, and Response Process received the lowest ratings (9, 2. and 2, respectively). Interrater reliability was good for Content, Internal Structure, and Relation to Other Variables (kappa range 0.52 to 0.96, all P values