An optimum method of displaying speakers' face images and captions for a real-time speech-to-caption system for the deaf

我们正在进行的实时语音字幕转换系统使用了语音识别技术和重复说话的方法，在日语-日语转换中，字幕的准确率约为97%。我们找到了用合适的方式显示字幕信息和说话人脸部运动图像的想法，以达到更高的理解力。结果表明，嘴部动作有助于提高对字幕的理解，“在说话者的面部图像之前显示字幕”的顺序也有助于提高对字幕的理解。

Our througoing real-time speech-to-caption system that using speech recognition technology with a "repeat-speaking" method showed that the accuracy of the captions is about 97% in Japanese-Japanese conversion. We found the idea to display information of captions and speaker's face movement images with a suitable way to achieve a higher comprehension. The results showed that the mouth movements contribute to improve the comprehension of the captions, and the sequence "to display the caption before the speaker's face image" improves the comprehension of the captions also.