Eye-Movement measurement while Learning Pronunciation Using English Pronunciation Practice CAI System Based on Sound and Mouth Movement

基于唇阅读技术，开发了“唇阅读”语音练习CAI系统。“唇阅读”使学习者能够通过观察模型说话者的嘴部运动来练习英语发音，并且允许比较他/她自己的嘴部运动，该嘴部运动由连接到计算机的摄像机同时显示在屏幕上。在使用“唇读阅读”后，本族语者可以很容易地识别学生的表现。“唇读阅读”通过纠正嘴部的动作，对改善发音是有效的。然而，对于每个声音，应该集中在嘴部的哪个部位并不总是明确的。因此，当他们观察计算机屏幕上的模型说话者的嘴时，眼睛相机被用来精确地测量母语者和学生的眼球运动。对眼动的各个方面进行了分析，并以数字方式传达数据。例如，一些学生没有观察到模型说话者的嘴，而是观察了面部的其他部位。有些学生不明白在何时、何地或如何正确地观察说话者的嘴部。为了吸引学生的注意力，并将其集中在发音和嘴部运动的正确方面，我们制作了一个新的计算机图形（CG）动画电影，强调嘴部运动的精确区域。使用这个新的CG动画系统，学生能够更有效地练习正确的发音。

The pronunciation practice CAI system called “Lip Reading" based on lip reading techniques has been developed. “Lip Reading" enables a learner to practice English pronunciation by looking at the model speaker's mouth movement and allows to compare his/her own mouth movement which is simultaneously shown on the screen by the video camera connected to a computer. After Using “Lip Reading", the native speakers could easily recognize the students' performance. “Lip Reading" is effective for improvements of pronunciation by correcting of mouth movements.However, it is not always clear which part of the mouth should be focused on for each sound. Therefore, eye cameras were used to precisely measure both native speakers' and students' eye movements as they observed the model speaker's mouth on the computer screen. A variety of aspects regarding eye movement were analyzed and the data was conveyed numerically. For example, some students did not observe the model speaker's mouth, but rather other parts of the face. Some students did not understand when, where or how to properly observe the model speaker's mouth.In order to capture the student's attention and focus it on the proper aspects of pronunciation and mouth movement, we produced a new computer graphics (CG) animation film emphasizing the precise areas of mouth movement. Using this new CG animation system, students were able to practice proper pronunciation much more effectively.