Does a rater's familiarity with a candidate's pronunciation affect the rating in oral proficiency interviews?
Does a rater's familiarity with a candidate's pronunciation affect the rating in oral proficiency interviews?
复制标题
DOI：
10.1177/0265532210393704
复制
发表时间：
2011-04-01
期刊：
影响因子：
4.1
通讯作者：
Dunn, Peter K.
中科院分区：
文献类型：
--
作者：
Carey, Michael D.;Mannell, Robert H.;Dunn, Peter K.
本研究探讨了影响口语测试中发音评估部分考官间信度的因素。我们假设，由于考官对非母语英语口音的接触程度，发音评分容易受到评估变化的影响。对汉语、韩语和印度英语三种测试候选中介语言的英语发音评分进行了评分间变异性分析。发音由99名国际英语语言测试系统（雅思）考官在五个地理位置分散的测试中心进行评分。审查员要么长时间接触考生的中介语言，要么不接触，要么很少接触。相当大比例的考官在长时间接触中间语时对发音的评价较高，而在没有或很少接触中间语时，对发音的评价较低。测试中心的位置也对发音评分有显著影响，独立于熟悉度变量，NNS评分者给来自本国的考生的评分高于那些来自非本国的考生。建议在设计口语测试和评分员培训时应考虑到对中介语音系的熟悉程度，并需要对测试中心的偏见和OPI评分员的音系判断和意识进行研究。
This study investigated factors that could affect inter-examiner reliability in the pronunciation assessment component of speaking tests. We hypothesized that the rating of pronunciation is susceptible to variation in assessment due to the amount of exposure examiners have to non-native English accents. An inter-rater variability analysis was conducted on the English pronunciation ratings of three test candidate interlanguages: Chinese, Korean, and Indian English. Pronunciation was rated by 99 International English Language Testing System (IELTS) examiners across five geographically dispersed test centres. The examiners had either prolonged exposure, or no, or little exposure to the interlanguage of the candidates. A significant proportion of examiners rated pronunciation higher when they had prolonged exposure, and lower when they had no, or little, exposure to the candidates' interlanguage. The location of the test centre also had a significant effect on the pronunciation rating, independent of the familiarity variable, with a significant proportion of NNS raters scoring candidates from their home country higher than those who were not. It is recommended that interlanguage phonology familiarity should be considered in the design of speaking tests and rater training and that research is required into test centre bias and the phonological judgements and awareness of OPI raters.