Code Localization in Programming Screencasts
Code Localization in Programming Screencasts
复制标题
DOI：
10.1007/s10664-019-09759-w
复制
发表时间：
2020-01
影响因子：
4.1
通讯作者：
Mohammad D. Alahmadi;Abdulkarim Khormi;Biswas Parajuli;Jonathan Hassel;S. Haiduc;Piyush Kumar
中科院分区：
文献类型：
--
作者：
Mohammad D. Alahmadi;Abdulkarim Khormi;Biswas Parajuli;Jonathan Hassel;S. Haiduc;Piyush Kumar
编程视频越来越受欢迎，并且经常被开发人员用作学习资源。这些屏幕视频中显示的源代码通常无法下载或复制粘贴。如果没有现成的代码，开发人员必须经常暂停视频来转录代码。这既耗时又降低了从视频中学习的效率。最近的方法是应用光学字符识别（OCR）技术从编程视频中自动提取源代码。然而，它们的主要限制之一是提取噪声，如菜单中的文本信息，包层次结构等，这是由于屏幕上代码位置的不精确近似。这会导致不正确的、不可用的代码。我们的目标是解决这一限制，并提出一种方法来显着提高编程屏幕视频中代码定位的准确性，从而实现更精确的代码提取。我们的方法使用卷积神经网络来自动预测图像中代码的确切位置。我们用一组从450个涵盖Java、c#和Python编程主题的屏幕视频中提取的框架来评估我们的方法。结果表明，我们的方法能够以94%的准确率检测包含代码的区域，并且我们的方法显着优于以前的工作。我们还表明，在我们的方法识别的代码区域上应用OCR，平均会导致97%的匹配与地面真实，相比之下，当将OCR应用于整个帧时，只有31%。
Programming screencasts are growing in popularity and are often used by developers as a learning source. The source code shown in these screencasts is often not available for download or copy-pasting. Without having the code readily available, developers have to frequently pause a video to transcribe the code. This is time-consuming and reduces the effectiveness of learning from videos. Recent approaches have applied Optical Character Recognition (OCR) techniques to automatically extract source code from programming screencasts. One of their major limitations, however, is the extraction of noise such as the text information in the menu, package hierarchy, etc. due to the imprecise approximation of the code location on the screen. This leads to incorrect, unusable code. We aim to address this limitation and propose an approach to significantly improve the accuracy of code localization in programming screencasts, leading to a more precise code extraction. Our approach uses a Convolutional Neural Network to automatically predict the exact location of code in an image. We evaluated our approach on a set of frames extracted from 450 screencasts covering Java, C#, and Python programming topics. The results show that our approach is able to detect the area containing the code with 94% accuracy and that our approach significantly outperforms previous work. We also show that applying OCR on the code area identified by our approach leads to a 97% match with the ground truth on average, compared to only 31% when OCR is applied to the entire frame.