Data extraction from videos for the analysis of behaviors in urban scenes

了解人、动物和车辆在城市场景中的行为和活动，对于更好地规划和了解如何建造道路、自行车道和公共场所（如公园）非常有用。对行为的良好理解可以使城市基础设施的规划更加安全，环保和高效。为了在城市规划中做出更好的决策，需要大量的数据。鉴于大量的数据，城市规划者需要能够自动提取有用数据的工具。这就是我们在本提案中要解决的大数据问题。我们希望通过自动检测感兴趣对象（OOI）并跟踪它们在场景中的运动来从视频中提取有用的数据。 现有的检测方法在各种情况下都失败了。例如，基于背景减除的方法是有限的，因为它们依赖于OOI和学习的场景分布之间的颜色对比度，并且它们假设相机是静止的。基于光流的方法要求OOI是移动的。最后，基于机器学习的多类检测器仅限于检测预定义的对象类，无法检测意外的OOI。对于所有这些方法，域适应是一个问题，因为它们通常在新的场景中表现不佳。因此，我们假设这些方法应该以某种方式结合在一起，并应采取措施，使它们可以很容易地适应新的情况。此外，跟踪方法对OOI检测的质量非常敏感。因此，我们希望联合研究OOI检测和跟踪，以提高这两个任务。 本提案的目标是研究并提出改进OOI检测方法的原始方法，以联合收割机从几种OOI方法检测到的OOI，找到使检测方法方便地适应新场景的方法，以及设计跟踪方法，该方法可以更好地利用OOI方法提供的信息，并且可以通过执行联合检测和跟踪来更好地处理检测错误。 本研究建议的独创性在于，我们研究通常单独研究的对象检测方法的融合，我们共同研究检测和跟踪，以提高这两项任务。我们将在本提案中开发的工具可用于任何视觉监控应用。

Understanding the behaviors and activities of people, animals and vehicles in urban scenes is very useful to better plan and understand how to build roads, bike paths and public places, such as parks. A good understanding of behaviors can make planning of urban infrastructures more secure, eco-friendly and efficient for its users. To make better decisions in urban planning, a large quantity of data is required. Given that large amount of data, urban planners need tools that can extract useful data automatically. This is the big data problem that we are addressing in this proposal. We want to extract useful data from videos by detecting automatically objects of interest (OOI) and tracking their motion in the scene. Existing detection methods all fail in various situations. For example, methods based on background subtraction are limited due to the fact that they rely on a color contrast between the OOI and the learnt scene distribution, and they assume that the camera is still. Methods based on optical flow require the OOIs to be moving. Finally, machine learning-based multiclass detectors are limited in detecting only predefined object classes and cannot detect unexpected OOIs. For all those methods, domain adaptation is an issue as they often underperform on new scenarios. Therefore, we hypothesize that these methods should be somehow combined together and care should be taken so that they can be easily adapted to new scenarios. Furthermore, tracking methods are very sensitive to the quality of the OOI detection. Therefore, we want to study jointly OOI detection and tracking to improve both tasks. The objectives of this proposal are to investigate and propose original approaches to improve OOI detection methods, to combine OOIs detected from several OOIs methods, to find ways to adapt detection methods conveniently to new scenarios, and to design tracking methods that can capitalize better on the information provided by the OOI methods and that can better handle detection errors by performing jointly detection and tracking. The originality of this research proposal is that we investigate the fusion of object detection methods that are usually studied separately and that we study jointly detection and tracking to improve both tasks. The tools that we will develop in this proposal can be used for any visual surveillance applications.