Development of a computer-assisted language learning system utilizing a speaker adaptation and a grammatical error modeling

1. 语音评估的改进我们开发了一种方法，通过应用语音识别技术来检测外语学习者的发音错误。我们专注于以下两个目标：以日本为母语的人学习英语和以韩国为母语的人学习日语。为了提高错误发音检测的准确性，我们开发了一种双语说话者适应方法，以适应英语和日语的hmm学习者。为了解决发音错误检测的严格程度取决于发音的语言语境的问题，我们开发了一种使用决策树检测发音错误的方法，该方法对以日语为母语的英语话语的检测准确率超过90%。语调和节奏的评估除了检测发音错误外，我们还开发了英语语音语调和节奏的评估方法。我们发现log F0、log幂及其导数是评价语调的良好特征。为了调整词间语调评价的严格程度，提出了一种基于决策树的词重要性因子估计方法。我们还发现，单词持续时间比是节奏评估的一个很好的特征。交互式呼叫系统的开发我们开发了一种从学习者的话语中检测语法错误的方法，用于交互式呼叫系统，使学习者能够通过与计算机的对话来学习外语。至于日语的学习，我们开发了一种方法，使用有限状态自动机来识别学习者的语音，其中应用了语法错误规则。至于英语学习，我们开发了一种方法，使用从使用语法错误规则自动生成的语料库中训练的n-gram语言模型。

1. Improvement of Pronunciation EvaluationWe developed a method to detect pronunciation mistakes made by a foreign language learner by applying a speech recognition technology. We focused on the following two targets: learning of English by Japanese native speakers and learning of Japanese by Korean native speakers. To improve the accuracy of detection of mispronunciations, we developed a bilingual speaker adaptation method to adapt both of English and Japanese HMMs to the learner. To solve a problem that the strictness of the detection of mispronunciations depends on the linguistic context of the pronunciation, we developed a method of detecting mispronunciations using a decision tree, which gave a detection accuracy of more than 90% on English utterances made by Japanese native speakers.2. Evaluation of Intonation and RhythmIn addition to the detection of mispronunciations, we developed methods to evaluate intonation and rhythm of the English utterances. We found that the log F0, log power and their derivatives were good features for the evaluation of intonation. To adjust the strictness of intonation evaluation from word to word, we introduced a method to estimate the word importance factors using a decision tree. We also found that a word duration ratio was a good feature for rhythm evaluation.3. Development of an interactive CALL systemWe developed a method of detecting grammatical errors from utterances of a learner for the application to an interactive CALL system that enables for a learner to learn a foreign language through dialogues with a computer. As for the learning of Japanese, we developed a method to recognize the learner's speech using a finite state automaton to which grammatical error rules were applied. As for the learning of English, we developed a method to use an n-gram language model trained from a corpus that was automatically generated using the grammatical error rules.