Data extraction from videos for the analysis of behaviors in urban scenes

点击翻译按钮获取中文摘要

Understanding the behaviors and activities of people, animals and vehicles in urban scenes is very useful to better plan and understand how to build roads, bike paths and public places, such as parks. A good understanding of behaviors can make planning of urban infrastructures more secure, eco-friendly and efficient for its users. To make better decisions in urban planning, a large quantity of data is required. Given that large amount of data, urban planners need tools that can extract useful data automatically. This is the big data problem that we are addressing in this proposal. We want to extract useful data from videos by detecting automatically objects of interest (OOI) and tracking their motion in the scene. Existing detection methods all fail in various situations. For example, methods based on background subtraction are limited due to the fact that they rely on a color contrast between the OOI and the learnt scene distribution, and they assume that the camera is still. Methods based on optical flow require the OOIs to be moving. Finally, machine learning-based multiclass detectors are limited in detecting only predefined object classes and cannot detect unexpected OOIs. For all those methods, domain adaptation is an issue as they often underperform on new scenarios. Therefore, we hypothesize that these methods should be somehow combined together and care should be taken so that they can be easily adapted to new scenarios. Furthermore, tracking methods are very sensitive to the quality of the OOI detection. Therefore, we want to study jointly OOI detection and tracking to improve both tasks. The objectives of this proposal are to investigate and propose original approaches to improve OOI detection methods, to combine OOIs detected from several OOIs methods, to find ways to adapt detection methods conveniently to new scenarios, and to design tracking methods that can capitalize better on the information provided by the OOI methods and that can better handle detection errors by performing jointly detection and tracking. The originality of this research proposal is that we investigate the fusion of object detection methods that are usually studied separately and that we study jointly detection and tracking to improve both tasks. The tools that we will develop in this proposal can be used for any visual surveillance applications.