Measuring Non-native Speakers’ Proficiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions
Measuring Non-native Speakers’ Proficiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions
复制标题
DOI：
10.3115/1609829.1609839
复制
发表时间：
2005-06
期刊：
影响因子：
--
通讯作者：
E. Sumita;F. Sugaya;Seiichi Yamamoto
中科院分区：
文献类型：
--
作者：
E. Sumita;F. Sugaya;Seiichi Yamamoto
本文提出了基于项目反应理论（IRT）的自动生成填空题和测试方法来测量英语水平。首先，该提案从给定的英语句子中生成FBQ。确定句子中空白的位置，该位置上的单词被认为是正确的选择。对于空白的错误选择的候选人是通过同义词典假设的。然后，通过Web对每个候选对象进行验证。最后，将空句、正确的选择和通过验证的不正确的选择放在一起构成FBQ。其次，非母语人士参加由这些fbq组成的测试，通过IRT来评估他们的熟练程度。我们的实验结果表明：(1)生成的问题加上IRT估计了非母语者的英语水平；(2)另一方面，以英语为母语的人几乎可以完美地完成测试；(3)在IRT中使用项目信息可以减少问题的数量。该方法为教师和测试人员提供了一种工具，减少了测试英语水平的时间和费用。
This paper proposes the automatic generation of Fill-in-the-Blank Questions (FBQs) together with testing based on Item Response Theory (IRT) to measure English proficiency. First, the proposal generates an FBQ from a given sentence in English. The position of a blank in the sentence is determined, and the word at that position is considered as the correct choice. The candidates for incorrect choices for the blank are hypothesized through a thesaurus. Then, each of the candidates is verified by using the Web. Finally, the blanked sentence, the correct choice and the incorrect choices surviving the verification are together laid out to form the FBQ. Second, the proficiency of non-native speakers who took the test consisting of such FBQs is estimated through IRT. Our experimental results suggest that: (1) the generated questions plus IRT estimate the non-native speakers' English proficiency; (2) while on the other hand, the test can be completed almost perfectly by English native speakers; and (3) the number of questions can be reduced by using item information in IRT. The proposed method provides teachers and testers with a tool that reduces time and expenditure for testing English proficiency.