Automatic Measuring of English Language Proficiency for Global Communications

这项研究的目的是(1)建立一个学习者语料库，其中包括不同英语水平的不同日语受试者翻译的英语句子和一些可用于主客观评价翻译质量的附加项目，以及(2)开发一种英语水平的自动测量方法。主要研究成果如下：1.作为自动测量英语水平的工具，我们建立了一个由500名日语受试者翻译的15万个英语句子的学习者语料库，其中日语来源句随机选自BTEC(基本旅行表达语料库)、50万个成对句子的英日平行语料库和初中和高中英语教材。每个源句设置10个参照句，以衡量被试的编辑距离、参照句与译文之间n元语法的相似度等特征。几种…译文的性质学习者语料库中更多的ES是由以英语为母语的人主观评价的，这些人经常说日语。对于受试者翻译的一些英语句子，英语本族语者的主观评价与BLEU等客观评价之间的相关性被计算出来，如果使用几十个英语句子，则确保了高度的相关性。这一实验结果保证了BLEU提出的客观评价机器翻译句子质量的方法可以用来衡量受试者的英语水平。为了获得较高的相关性分数，对不同的翻译目标句选择方法进行了比较，提出了一种以较少的句子数获得高相关性的选择方法。学习者语料库除了用于英语水平的自动测量外，还有望用于各种应用。其他应用之一是创建一个概率语言模型，该模型表示英语母语者和日语使用者的话语在词汇和句法上的差异。为了验证这一思想，我们建立了一个二元语法和三元语法的概率语言模型，并用BTEC语料库进行了线性内插，BTEC语料库是一个大型标准英语句子语料库。使用线性内插语言模型的实验结果表明，该语言模型比单独使用BTEC训练的语言模型提供了更好的单词准确率。这表明，用学习者语料库训练的模型和用大量英语本族语语料库训练的模型之间的语言模型适应可以有效地补偿本族语者和第二语言者在词汇和句法特征上的不匹配。较少

Purposes of this re arch are (1) creation of a learner corpus which contains English sentences translated by various Japanese subjects of different English proficiency and some additional items available for subjective and objective evaluation of translation quality, and (2) development of an automatic measuring method of English proficiency. Main research results are as follows ;1. As tools for automatic measuring English proficiency, we created a learner corpus of 150,000 English sentences translated by 500 Japanese subjects, of which Japanese source sentences were randomly selected from BTEC (Basic Travel Expression Corpus), English-Japanese parallel corpus of 500,000 paired sentences, and textbooks on English for junior and senior high schools. Ten reference sentences per each source sentence were made to measure some features such as edit-distance, similarity of n-grams between the reference sentences and translated sentences by the subject. Qualities of some of translated sentenc … More es in the learner corpus were subjectively evaluated by native English speakers who are frequent speaker of Japanese.2. Correlation were calculated between subjective evaluation by the English native speakers and objective evaluation such as BLEU for some English sentences translated by the subjects, and high correlation was assured if dozens of English sentences were utilized. This experimental result assured that the objective evaluation method of BLEU, which was proposed to objectively evaluate quality of sentences translated with MT technologies, can be used for measuring English proficiency of the subjects.3. Different method for selecting target sentences for translation were checked necessary number of target sentences in order to obtain a high correlation score, and one selection method for obtaining high correlation with smaller number sentences was proposed.4. The learner corpus is expected to be used for various applications besides automatic measuring of English proficiency. One of the other applications is to create a probabilistic language model which represents lexical and syntactical difference between utterances by native English speakers and Japanese. In order to verify the idea, we created a probabilistic language model such bi-gram and tri-gram and linearly interpolated them with a language models which were trained with BTEC corpus, a large text corpus of canonical English sentences. The experimental results using the linearly interpolated language model shown that the language model provided better word accuracy than a language model trained with BTEC alone. This shows that the language model adaptation between a model trained with the learner corpus and a model trained with large corpus of native English is effective for compensating for the mismatch between the lexical and syntactical characteristics of native speakers and second language speakers. Less