Automated Detection of Handwritten Whiteboard Content in Lecture Videos for Summarization
Automated Detection of Handwritten Whiteboard Content in Lecture Videos for Summarization
复制标题
DOI：
10.1109/icfhr-2018.2018.00013
复制
发表时间：
2018-08
期刊：
影响因子：
--
通讯作者：
B. Kota;Kenny Davila;Alexander Stone;S. Setlur;V. Govindaraju
中科院分区：
文献类型：
--
作者：
B. Kota;Kenny Davila;Alexander Stone;S. Setlur;V. Govindaraju
在线课程视频是世界各地学生的宝贵资源。根据内容查找视频的能力可能会使它们更加有用。用于自动提取该内容的方法减少了使索引和检索这样的视频成为可能所需的手动工作量。我们采用了一种基于深度学习的场景文本检测方法，用于检测教学视频中的手写文本、数学表达式和草图。我们检测白板上的手写元素，以生成讲座中所有内容随时间变化的摘要，同时还处理由于讲师运动而被遮挡的内容。我们在公开的CIMMath讲座视频数据集上进行训练和测试，并根据摘要帧的数量以及测试讲座视频集中所有白板内容的召回率和精度来评估我们的框架。我们发现，我们的方法提高了最先进的精确度，同时也有可能提高召回率。我们通过为已发布的手写白板内容提供基于时间戳的、语义上有意义的边界框注释，增加了EMPath数据集中现有的地面实况。
Online lecture videos are a valuable resource for students across the world. The ability to find videos based on their content could make them even more useful. Methods for automatic extraction of this content reduce the amount of manual effort required to make indexing and retrieval of such videos possible. We adapt a deep learning based method for scene text detection, for the purpose of detection of handwritten text, math expressions and sketches in lecture videos. We detect handwritten elements on the whiteboard to generate a summary of all content over time in the lecture, while also dealing with occluded content due to motion of the lecturer. We train, test on the publicly available AccessMath lecture video dataset and evaluate our framework on the basis of number of summary frames, as well as recall and precision of all whiteboard content in the set of test lecture videos. We found that our method increases the precision of the state-of-the-art while there is potential to increase recall as well. We have added to the existing ground truth in the AccessMath dataset by providing timestamp-based, semantically meaningful bounding box annotations for the handwritten whiteboard content, which has been released.