基于深度学习的外语视频个性化字幕呈现及摘要生成方法研究

由于缺乏外语语言环境，学习者经常通过观看原声视频及附带字幕来弥补这一不足。然而，现有视频内容和字幕仅仅是单向地传递信息，学习者个体学习行为和个性化学习数据无法被利用，并且学习者的学习效果不能得到反馈。针对外语视频学习中的个性化学习需求，本项目拟利用大规模人眼视线数据集和深度卷积神经网络等深度学习技术，基于学习者实时视线估计和个性化学习数据，对字幕的呈现位置、内容组织和可视化及视频摘要的生成方法开展研究。研究内容主要包括：基于学习者实时视线方向估计的字幕呈现位置研究；基于学习者个体学习数据的字幕内容组织及可视化方法研究；外语视频的个性化摘要生成技术研究及新字幕呈现方式和新生成的视频摘要对学习效果影响的客观评价方法研究。本项目拟通过对个性化字幕呈现、视频摘要生成方法和评价体系的研究，提高学习者的外语学习效果。此外，项目成果对丰富自适应个性化辅助学习理论、方法和工具具有重要意义，应用前景广泛。

Due to the lack of language atmosphere, many learners learn foreign languages by watching videos subtitles. However, current videos and subtitles only convey information to learners with no responses. They also did not take personalized learning behavior and data into consideration. In this project, by using a big human eye image dataset and Convolutional Neural Network (CNN) technique, real-time eye gaze estimation and personalized learning data are adopted to research on the personalized subtitle position/content visualization and video abstraction problem. The core research contents include: subtitle positioning based on real time eye gaze estimation; subtitle contents organization and visualization based on personalized learning data; personalized video abstraction; quantitative evaluation on the relationship between proposed subtitles/abstraction and learning effect and further suggestions for learners. The research results of this project will improve foreign language learning effects. The research findings will also enrich the theory of self-adapted personalized learning, and its tools and methods. Also, the findings can be applied to plenty of areas in the future.