Integrating object segmentation for robust object tracking

视频对象跟踪是一个活跃的研究领域，其应用范围广泛，包括增强现实、自主导航系统（无人机或自动驾驶汽车）、人机交互、超链接社交视频或运动员表现监测。跟踪在原始视频信号上执行，通常从单个摄像机捕获；可以跟踪不同类型的对象，如人、运动员、车辆、动物或船舶。在真实世界的视频信号中跟踪对象有许多挑战，例如由于规模变化、遮挡、衔接、变形和多个对象的相互作用而导致的对象外观变化。***最近的目标跟踪算法已经很好地推进了这些挑战中的一些主题。在本提案中，我将解决以下研究挑战，以推进最先进的单目标跟踪：1)目标跟踪的漂移：当跟踪器偏离目标时该怎么办以及如何检测这种漂移？2)可变对象特征：如何处理跟踪过程中变化较大的特征，如颜色直方图等？3)物体的尺度变化：我们是可以显式地建模尺度变化，还是隐式地建模，比如通过与物体分割和检测相结合？***为了检测特定跟踪器的漂移，我建议通过监测其潜在参数随时间的变化来分析其内部状态。对于与跟踪器无关的漂移检测，我将整合目标分割和abjectness（图像区域是目标的可能性）度量。为了纠正漂移，我将在漂移发生时将物体分割和物体检测（定位）集成到跟踪中。分割的使用是由人类视觉系统的最新发现激发的，它似乎依赖于物体的空间和时间间隔来进行有效的跟踪。我将特别使用对象分割和定位来探索视频对象之间的现在-未来（或时空）交互。***为了解决视频序列中变量特征的问题，我将使用机器学习，例如使用支持向量机的在线多核学习，来在线确认或拒绝特征。***我将通过监测物体在空间和时间上的状态来处理物体的尺度变化，通过单应变换，并将深度作为一个附加功能。***许多数据集、真实数据和目标跟踪指标的可用性将使新方法的有效评估成为可能。我预计开发的方法将补充现有的在线学习方法，为跟踪辅助视频应用程序增加额外的健壮性。人们对广泛应用（鲁棒）但快速的目标跟踪的兴趣很大。凭借我20多年的研究和行业经验，我有信心很好地推进知识并将其转移到加拿大视频技术行业的相关部门。

Video object tracking is an active research field with applications diverse as augmented reality, self-navigation systems (drones or self-driving cars), human-computer interactions, hyper-linked social video, or athlete performance monitoring. Tracking is performed on raw video signals, captured typically from a single camera; different types of objects can be tracked, such as persons, athletes, vehicles, animals, or ships. Tracking of objects across real-world video signals has many challenges such as variations in object appearance due to scale change, occlusion, articulation, deformation, and interactions of multiple objects. ***Recent algorithms to object tracking have well advanced the topic in some of these challenges. In this proposal, I will address the following research challenges to advance the state-of-the-arts to single object tracking: 1) drift of object tracking: what to do when a tracker drifts from the target and how to detect such drift? 2) variable object features: how to deal with features, such as color histogram, that significantly vary during tracking? 3) scale change of objects: can we explicitly model scale change or rather implicitly such as through the integration with object segmentation and detection?***To detect drifts of a specific tracker, I propose to analyze its internal state by monitoring its latent parameters over time. For tracker-independent drift detection, I will integrate object segmentation and abjectness (likelihood an image region is an object) measures. To correct drifts, I will integrate object segmentation and object detection (localization) into tracking as drift happens. The use of segmentation is motivated by recent discoveries on the human visual system which seems to rely on spatial and temporal spacing of the objects for effective tracking. I will specifically use object segmentation and localization to explore present-future (or space-time) interaction between video objects. ***To solve the problem of a variable feature across a video sequence, I will use machine learning, e.g., online multi-kernel learning with support vector machines, to online confirm or reject features. ***I will handle scale change of objects through monitoring of object states in space and time, through homography transformation, and depth as an added feature. ***Availability of many datasets, ground-truth data, and metrics of object tracking will enable effective evaluation of the new approaches. I anticipate that developed methods will complement existing online learning approaches, adding an extra level of robustness to tracking-assisted video applications. Interest in widely-applicable (robust) but fast object tracking is large. With my over 20 years of research and industrial experience, I am confident to well advance knowledge and transfer it to related sectors of the Canadian video technology industries.