Using eye-tracking data for automatic film comic creation
Using eye-tracking data for automatic film comic creation
复制标题
DOI：
10.1145/2168556.2168640
复制
发表时间：
2012-03
期刊：
影响因子：
--
通讯作者：
M. Toyoura;Tomoya Sawada;Mamoru Kunihiro;Xiaoyang Mao
中科院分区：
文献类型：
--
作者：
M. Toyoura;Tomoya Sawada;Mamoru Kunihiro;Xiaoyang Mao
电影连环画是以连环画的形式表现电影故事的艺术作品。它使用电影的图像作为面板。语言信息，如对话和叙述，用单词气球表示。电影漫画创作中的一个关键问题是如何选择对传达电影故事有意义的图像。图像的这种重要性本质上是语义和上下文相关的，因此，纯粹基于图像分析的技术通常无法产生良好的结果。另一方面，单词气球排列不仅需要理解图像的语义，还需要理解言语信息，这除了电影剧本可用的情况外是困难的。本文介绍了一种新的尝试，使用眼动跟踪数据的自动创建的电影漫画从电影。分析眼睛运动的模式来检测场景的变化，并使用注视信息来自动找到插入和引导单词气球的位置。实验结果表明，与仅利用图像特征的方法相比，该方法能大大提高重要图像的选择能力，实现气球的自动排列。
A film comic is a kind of art work representing a movie story as a comic. It uses the images of the movie as panels. Verbal information such as dialogue and narrations is represented in word balloons. A key issue in creating film comics is how to select images which are significant in conveying the story of the movie. Such significance of images is inherently semantic and context-dependent and hence, technologies purely based on image analysis usually fail to produce good results. On the other hand, the word balloon arrangement requires understanding not only the semantic of images but also the verbal information, which is difficult except for the case the script of the movie is available. This paper describes a new attempt to use eye-tracking data for the automatic creation of a film comic from a movie. Patterns of eye movement are analyzed for detecting the change of scenes and gaze information is used for automatically finding the location for inserting and directing the word balloons. Our experiments showed that the proposed technique can largely improve the selection of significant images compared with the method using image features only and realize the automatic balloon arrangement.