Assessment of Psychology Competencies in Field Placements: Standardized Vignettes Reduce Rater Bias
Assessment of Psychology Competencies in Field Placements: Standardized Vignettes Reduce Rater Bias
复制标题
DOI：
10.1037/a0031617
复制
发表时间：
2013-05-01
影响因子：
2
通讯作者：
Knight, Roslyn
中科院分区：
文献类型：
--
作者：
Gonsalvez, Craig J.;Bushnell, John;Knight, Roslyn
指导者对心理学实习生在现场环境中的能力进行评定是培训评估的重要组成部分。关于这些评估的有效性几乎没有系统的研究，但现有的证据表明我们有问题！监管者的判断可能会受到系统性偏见的影响，这些偏见对评估的可信度构成严重威胁。目前的研究是六所大学之间的研究合作的一部分，该研究致力于开发和评估一种新的方法，即使用小插曲来对抗传统评级量表的结果。每个小插图都经过了严格的同行评审和修订，然后由一组专家为最终小插图分配校准分数。汇编了一个小插曲（n = 41）的目录，代表跨几个发展阶段的各个领域的能力。大学和外地主管使用传统的评级量表和插图匹配程序（VMP），以评估能力，最终安置。一项试点研究（n = 20）和一项后续研究（n = 57）的数据表明，与传统的评级量表相比，VMP减少了宽大和光环偏见。VMP有潜力改善实地安置能力评估的结果，值得进一步研究和开发。
Supervisors' ratings of psychology trainees' competence in field settings are a critical component of training assessment. There has been little systematic research regarding the validity of these assessments, but the available evidence suggests we have a problem! Supervisors' judgments may be affected by systemic biases that pose a serious threat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method the use of vignettes against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer-review and revisions, before final vignettes were assigned calibration scores by a group of experts. A catalogue of vignettes (n = 41) that represent various domains of competence across several developmental stages was compiled. University and field supervisors used the conventional rating scale and the vignette-matching procedure (VMP) to evaluate competencies at end-placement. Data from a pilot (n = 20) and a follow-up study (n = 57) suggest that compared with a conventional rating scale, the VMP reduced leniency and halo biases. The VMP has the potential to improve outcomes of competency assessments in field placements and merits further research and development.