Workplace-based assessment: effects of rater expertise.
Workplace-based assessment: effects of rater expertise.
复制标题
DOI：
10.1007/s10459-010-9250-7
复制
发表时间：
2011-05
影响因子：
4
通讯作者：
Muijtjens, A. M. M.
中科院分区：
文献类型：
--
作者：
Govaerts, M. J. B.;Schuwirth, L. W. T.;Van der Vleuten, C. P. M.;Muijtjens, A. M. M.
关键词：
传统的心理测量评估方法倾向于只关注评估结果的定量属性。这可能会限制更有意义的教育方法，以工作场所为基础的评估（WBA）。基于认知的工作行为分析模型认为，评估结果是由评估者的认知过程决定的，这与医学等专业领域的推理、判断和决策非常相似。本研究探讨了认知过程的基础上的判断和决策时，观察性能在临床工作场所的评分。它特别关注评分经验的差异如何影响评分员的信息处理。口头协议分析被用来调查如何有经验的和无经验的评分员选择和使用观察数据，以达到判断和决定学员的表现在临床工作场所。有经验和无经验的评分员之间的差异进行了评估方面花费的时间在信息分析和学员表现的代表性;性能评分;和信息处理-使用基于定性的定量分析的口头数据。结果表明，专家新手的差异所需的时间代表学员的表现，根据复杂性的评级任务。专家们更加关注评估背景下的特定情境线索，他们对观察到的行为产生了（显着）更多的解释和更少的文字描述。评分无显著差异。总的来说，我们的研究结果似乎与其他专家研究的结果一致，支持临床工作中基于认知的评估模型的理论。WBA的影响进行了讨论。
Traditional psychometric approaches towards assessment tend to focus exclusively on quantitative properties of assessment outcomes. This may limit more meaningful educational approaches towards workplace-based assessment (WBA). Cognition-based models of WBA argue that assessment outcomes are determined by cognitive processes by raters which are very similar to reasoning, judgment and decision making in professional domains such as medicine. The present study explores cognitive processes that underlie judgment and decision making by raters when observing performance in the clinical workplace. It specifically focuses on how differences in rating experience influence information processing by raters. Verbal protocol analysis was used to investigate how experienced and non-experienced raters select and use observational data to arrive at judgments and decisions about trainees’ performance in the clinical workplace. Differences between experienced and non-experienced raters were assessed with respect to time spent on information analysis and representation of trainee performance; performance scores; and information processing––using qualitative-based quantitative analysis of verbal data. Results showed expert-novice differences in time needed for representation of trainee performance, depending on complexity of the rating task. Experts paid more attention to situation-specific cues in the assessment context and they generated (significantly) more interpretations and fewer literal descriptions of observed behaviors. There were no significant differences in rating scores. Overall, our findings seemed to be consistent with other findings on expertise research, supporting theories underlying cognition-based models of assessment in the clinical workplace. Implications for WBA are discussed.
登录
查看更多内容
影响因子：
2.5
作者：
CHI, MTH;FELTOVICH, PJ;GLASER, R
通讯作者：
GLASER, R
影响因子：
2.9
作者：
Kreiter, CD;Ferguson, KJ
通讯作者：
Ferguson, KJ
影响因子：
7.4
作者：
Ericsson, KA
通讯作者：
Ericsson, KA
影响因子：
6
作者：
Eva, KW
通讯作者：
Eva, KW
影响因子：
6
作者：
HOBUS, PPM;SCHMIDT, HG;PATEL, VL
通讯作者：
PATEL, VL