Aligning plot synopses to videos for story-based retrieval
Aligning plot synopses to videos for story-based retrieval
复制标题
DOI：
10.1007/s13735-014-0065-9
复制
发表时间：
2015-03
影响因子：
5.6
通讯作者：
Makarand Tapaswi;M. Bäuml;R. Stiefelhagen
中科院分区：
文献类型：
--
作者：
Makarand Tapaswi;M. Bäuml;R. Stiefelhagen
我们提出了一种方便搜索电视剧剧集情节的方法。为此，我们使用了视频中传达的故事的人写的、众包的描述-情节提要。我们从维基百科等网站上获得了这样的概要，并提出了各种方法来将情节中的每一句话与视频中的镜头对齐。因此，基于语义故事的视频检索问题被转换为更简单的基于文本的搜索。最后，我们返回与句子对齐的镜头集作为对应于查询的视频片段。对齐是通过首先通过诸如剧情概要和字幕之间的角色身份和关键字匹配等线索来计算每个镜头和句子之间的相似性分数来执行的。然后，我们将比对问题描述为一个优化问题，并使用动态规划有效地解决它。我们在电视连续剧《吸血鬼杀手巴菲》第五季中评估了我们的方法，并在故事事件的对齐和检索方面显示了令人鼓舞的结果。
We propose a method to facilitate search through the storyline of TV series episodes. To this end, we use human written, crowdsourced descriptions—plot synopses—of the story conveyed in the video. We obtain such synopses from websites such as Wikipedia and propose various methods to align each sentence of the plot to shots in the video. Thus, the semantic story-based video retrieval problem is transformed into a much simpler text-based search. Finally, we return the set of shots aligned to the sentences as the video snippet corresponding to the query. The alignment is performed by first computing a similarity score between every shot and sentence through cues such as character identities and keyword matches between plot synopses and subtitles. We then formulate the alignment as an optimization problem and solve it efficiently using dynamic programming. We evaluate our methods on the fifth season of a TV series Buffy the Vampire Slayer and show encouraging results for both the alignment and the retrieval of story events.