A Deep Reinforcement Learning Framework for Identifying Funny Scenes in Movies
A Deep Reinforcement Learning Framework for Identifying Funny Scenes in Movies
复制标题
DOI：
10.1109/icassp.2018.8462686
复制
发表时间：
2018-04
期刊：
影响因子：
--
通讯作者：
Haoqi Li;Naveen Kumar;Ruxin Chen;P. Georgiou
中科院分区：
文献类型：
--
作者：
Haoqi Li;Naveen Kumar;Ruxin Chen;P. Georgiou
本文提出了一种新颖的深度强化学习（RL）框架，用于使用视频流中检测到的面部图像作为输入，根据情感对电影场景进行分类。从视频中提取情感信息是一项具有挑战性的任务，它需要调制复杂的视觉和时间表示，与人类感知和信息集成的复杂方面交织在一起。这也使得收集大型注释语料库变得困难，限制了监督学习方法的使用。我们提出了一种基于强化学习的替代学习框架，它能够容忍标签稀疏性，并且可以轻松地以在线方式利用任何可用的地面事实。我们使用这个修改后的强化学习模型对电影场景剪辑数据集上的场景是否有趣进行二元分类。结果表明，我们的模型在 2-3 分钟长的电影场景上的正确预测率为 72.95%，而在较短的场景上，预测准确率为 84.13%。
This paper presents a novel deep Reinforcement Learning (RL) framework for classifying movie scenes based on affect using the face images detected in the video stream as input. Extracting affective information from the video is a challenging task modulating complex visual and temporal representations intertwined with the complex aspects of human perception and information integration. This also makes it difficult to collect a large annotated corpus restricting the use of supervised learning methods. We present an alternative learning framework based on RL that is tolerant to label sparsity and can easily make use of any available ground truth in an online fashion. We employ this modified RL model for the binary classification of whether a scene is funny or not on a dataset of movie scene clips. The results show that our model correctly predicts 72.95% of the time on the 2–3 minute long movie scenes while on shorter scenes the accuracy obtained is 84.13%.