Deep Learning of Surveillance Video (EPSRC iCASE BAE)

深度信念网络是各种模式识别/机器学习任务的流行方法，通常优于传统的计算机分析方法和人类。有一个新兴的静态图像分析文献（例如http://image-net.org/about-publication）。然而，视频分析直到最近才开始被研究，最突出的是Facebook C3D工作，它表明将问题作为3D（随时间推移的2D图像）卷积问题优于更传统的方法，并且还用光流数据增强了原始像素数据。固定宽度窗口的一个问题是用于捕获和建模时间动态。虽然这可能适用于C3D中解决的任务（例如体育视频分类），但在许多其他领域可能存在限制。最近在BMVC2015上的一篇论文也讨论了视频异常检测的相关领域。该项目将建立在最近对如何在监控视频中检测打斗的调查基础上，其中活动发生在不同的持续时间和多尺度（随着时间的推移）已经被开发出来以解释这些特征。标准的图像特征识别方法在夜间光照、变分辨率、拥挤场景等条件下不能很好地工作，因此开发了基于纹理的时间特征来解决这一问题。比较一种深度信念网络方法会很有趣。该项目还将更广泛地关注异常事件的检测，主要是开发警报系统，帮助操作员应对现代监控室中数百个监控摄像头的挑战。一个扩展将是调查方法来编目或总结视频记录只有重要的事件或关键帧（序列），其中事件的变化或已被确定为异常或包含潜在的有趣的活动。主要的研究挑战是定义如何在深度学习方法中最好地处理不同持续时间的时间数据。

Deep Belief Networks are popular methods for a variety of pattern recognition/machine learning tasks often outperforming conventional computer analysis methods and humans. There is an emerging literature in static image analysis (e.g. http://image-net.org/about-publication). However, analysis of video is only recently beginning to be studied, the most prominent being the Facebook C3D work, which shows that posing the problem as a 3D (2D imagery over time) convolutional problem outperforms more traditional methods, and also augmented the raw pixel data with optical flow data. One issue that remains a fixed width window is used to capture and model the temporal dynamics. Whilst this might be appropriate for the tasks addressed in C3D (e.g. sports video classification) this may have limitations in many other domains. A recent paper at BMVC2015 also addresses the related area of anomaly detection in video.The project will build on recent investigation of how to detect fights in surveillance video, where activity occurs at varying duration and multi-scale (over time) have been developed to account for such features. Standard image feature recognition methods did not work well under challenge (night-time illumination, variable resolution, crowded scenes) and imaging conditions and texture based temporal features were developed to address the problem. It would interesting to compare a deep belief network approach.This project will also look more generally at detecting anomalous events, principally to develop alert systems assisting human operators to deal with the challenge of many hundreds of surveillance camera feeds in modern surveillance rooms. One extension would be to investigate methods to catalogue or summarise video - logging only important events or key frames (sequences) where events change or have been identified as anomalous or contain potential interesting activities.The main research challenge is to define how best to process temporal data of varying duration within a deep learning approach.