Model design for analyzing ultra-low framerate surveillance videos

点击翻译按钮获取中文摘要

The overarching objective of this project is to develop the next generation of automatic scene understanding techniques applied to ultra-low frame-rate traffic videos (less than 1 frame per second). The goals are to (1) localize and recognize vehicles pictured by live traffic cameras, and (2) develop compression techniques that shall allow the deployment of video analytics methods directly onto cameras. My research program aims at the integration of these two components into real-time video surveillance systems that will allow the gathering of high-level traffic statistics.******One important challenge is the fact that traffic images come with large illumination variations, compression artifacts, arbitrary vehicle orientation and scale, poor resolution, various weather conditions and inter-class similarities. As such, we shall leverage the power of deep convolutional neural networks (CNN) combined with large datasets. However, the plain use of off-the-shell CNNs is ill-suited to reach the aforementioned goals. First, state-of-the-art CNN are too large and require too much processing power to be stored on today's surveillance cameras. Second, by their very nature, traffic datasets are heavily imbalanced in the type of vehicles they contain. As such, they contain 100 times more cars than other vehicles (like motorcycles and road trains) which cause machine-learning methods to have a low recall on certain category of vehicles. Furthermore, the dataset that will be gathered for this project is so large (4 to 5 million images) that each vehicle in each image cannot be manually outlined. Instead, the images come with meta-information containing the time of the day, the type of vehicles present in the scene, the overall density, the global context, and a 2d dot on each vehicle. This research program thus aims at designing end-to-end trainable systems adapted to the weekly-annotated and unbalanced nature of traffic data as well as compression techniques for deploying it onto surveillance cameras. This includes innovative segmentation and localization techniques, network hyperparameters estimation through reinforcement learning, and optimization methods with sparsity and orthogonality constraints. These models have the potential to be transferred rapidly to Canadian companies and greatly improve daily traffic conditions. This program will also train 2 M.Sc and 3 Ph.D. for a demanding, highly multidisciplinary field.