Eye-Movement measurement while Learning Pronunciation Using English Pronunciation Practice CAI System Based on Sound and Mouth Movement

基于唇读技术，开发了语音练习CAI系统“唇读”。“唇读”可以让学习者通过观察说话者的嘴部运动来练习英语发音，并可以通过连接到电脑上的摄像机同时在屏幕上显示自己的嘴部运动。使用“唇读”后，母语人士可以很容易地识别学生的表现。“唇读”是通过纠正嘴部动作来改善发音的有效方法。然而，我们并不总是清楚每个音应该集中在嘴巴的哪个部位。因此，当母语者和学生在电脑屏幕上观察模范说话者的嘴巴时，眼睛摄像头被用来精确测量他们的眼球运动。对眼动的各个方面进行了分析，并对数据进行了数值传递。例如，一些学生没有观察演讲者的嘴巴，而是观察脸部的其他部分。有些学生不明白何时、何地或如何正确观察模范说话人的嘴部。为了吸引学生的注意力，并将其集中在发音和嘴部运动的适当方面，我们制作了一个新的计算机图形（CG）动画电影，强调嘴部运动的精确区域。使用这个新的CG动画系统，学生们能够更有效地练习正确的发音。

The pronunciation practice CAI system called “Lip Reading" based on lip reading techniques has been developed. “Lip Reading" enables a learner to practice English pronunciation by looking at the model speaker's mouth movement and allows to compare his/her own mouth movement which is simultaneously shown on the screen by the video camera connected to a computer. After Using “Lip Reading", the native speakers could easily recognize the students' performance. “Lip Reading" is effective for improvements of pronunciation by correcting of mouth movements.However, it is not always clear which part of the mouth should be focused on for each sound. Therefore, eye cameras were used to precisely measure both native speakers' and students' eye movements as they observed the model speaker's mouth on the computer screen. A variety of aspects regarding eye movement were analyzed and the data was conveyed numerically. For example, some students did not observe the model speaker's mouth, but rather other parts of the face. Some students did not understand when, where or how to properly observe the model speaker's mouth.In order to capture the student's attention and focus it on the proper aspects of pronunciation and mouth movement, we produced a new computer graphics (CG) animation film emphasizing the precise areas of mouth movement. Using this new CG animation system, students were able to practice proper pronunciation much more effectively.