Using Inter-Rater Discourse to Trace the Origins of Disagreement: Towards Collective Reflective Practice in L2 Assessment
Using Inter-Rater Discourse to Trace the Origins of Disagreement: Towards Collective Reflective Practice in L2 Assessment
复制标题
使用评估者间的话语来追踪分歧的根源：走向第二语言评估中的集体反思实践
DOI：
10.1177/0033688220977373
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
J. Matthews
中科院分区：
文献类型：
--
作者：
J. Matthews
本文探讨了如何利用评分者话语分析来支持二语评估中的集体反思实践。为了证明这一点，本文提出了一个有经验的语言教师之间的对话的集中案例，因为他们谈判的评估决定的L2书面文本。特别令人感兴趣的是围绕评分员最不一致的评估决定的话语，在这种情况下，这些决定与任务完成有关。主位分析表明，评分员话语主要集中在明确的客观因素，主要是第二语言文本和评分量表，但是，评分员话语也集中在更主观的，以评分员为中心的因素。围绕这些以评级者为中心的因素的讨论往往是识别和解决评级分歧的核心。本文认为，语言评估的主观维度需要更直接和系统地反映在语言教学环境中，而对评分者话语的分析，特别是关注评分者之间分歧点的话语，提供了一个有价值的机制，以促进这一点。
This article explores how the analysis of inter-rater discourse can be used to support collective reflective practice in second language (L2) assessment. To demonstrate, a focused case of the discourse between two experienced language teachers as they negotiate assessment decisions on L2 written texts is presented. Of particular interest was the discourse surrounding the raters’ most divergent assessment decisions, which in this case were those relating to Task Achievement. Thematic analysis indicated that rater discourse predominantly focused on explicit objective factors, primarily the L2 texts and the rating scale; however, rater discourse also focused on more subjective, rater-centred factors. The discourse surrounding these rater-centred factors was often central to the identification and resolution of rating disagreements. The paper argues that the subjective dimension of language assessment needs to be more directly and systematically reflected upon in language teaching contexts and that analysis of rater discourse, especially discourse focused on points of disagreement between raters, provides a valuable mechanism to facilitate this.