Integrating object segmentation for robust object tracking

视频对象跟踪是一个活跃的研究领域，应用广泛，如增强现实、自动导航系统(无人机或自动驾驶汽车)、人机交互、超链接社交视频或运动员表现监测。跟踪是对通常从单个摄像机捕获的原始视频信号执行的；可以跟踪不同类型的对象，例如人、运动员、车辆、动物或船只。在真实世界的视频信号中跟踪对象具有许多挑战，例如由于多个对象的比例变化、遮挡、清晰度、变形和相互作用而导致对象外观的变化。 最近的目标跟踪算法在其中一些挑战中很好地推进了这一主题。在这个提案中，我将解决以下研究挑战，以将最新技术发展到单个目标跟踪：1)目标跟踪的漂移：当跟踪器偏离目标时该做什么，以及如何检测这种漂移？2)可变的目标特征：如何处理跟踪过程中显著变化的特征，如颜色直方图？3)目标的尺度变化：我们是否可以显式地模拟尺度变化，或者更确切地说，通过与对象分割和检测的集成来隐式建模？ 为了检测特定跟踪器的漂移，我建议通过监测其随时间的潜在参数来分析其内部状态。对于跟踪器无关的漂移检测，我将结合对象分割和差异性(图像区域是对象的可能性)度量。为了纠正漂移，我将在发生漂移时将对象分割和对象检测(定位)集成到跟踪中。分割的使用是由人类视觉系统的最新发现推动的，人类视觉系统似乎依赖于对象的空间和时间间隔来进行有效的跟踪。我将专门使用对象分割和定位来探索视频对象之间的现在-未来(或空间-时间)交互。 为了解决视频序列中特征可变的问题，我将使用机器学习，例如，使用支持向量机的在线多核学习来在线确认或拒绝特征。 我将通过监控对象在空间和时间上的状态，通过单应变换和深度作为附加功能来处理对象的比例变化。 许多数据集、地面真实数据和目标跟踪指标的可用性将使新方法的有效评估成为可能。我预计，开发的方法将补充现有的在线学习方法，为跟踪辅助视频应用程序增加额外的健壮性。对广泛适用(稳健)但快速的目标跟踪的兴趣很大。凭借我20多年的研究和行业经验，我有信心很好地提升知识，并将其转移到加拿大视频技术行业的相关部门。

Video object tracking is an active research field with applications diverse as augmented reality, self-navigation systems (drones or self-driving cars), human-computer interactions, hyper-linked social video, or athlete performance monitoring. Tracking is performed on raw video signals, captured typically from a single camera; different types of objects can be tracked, such as persons, athletes, vehicles, animals, or ships. Tracking of objects across real-world video signals has many challenges such as variations in object appearance due to scale change, occlusion, articulation, deformation, and interactions of multiple objects. Recent algorithms to object tracking have well advanced the topic in some of these challenges. In this proposal, I will address the following research challenges to advance the state-of-the-arts to single object tracking: 1) drift of object tracking: what to do when a tracker drifts from the target and how to detect such drift? 2) variable object features: how to deal with features, such as color histogram, that significantly vary during tracking? 3) scale change of objects: can we explicitly model scale change or rather implicitly such as through the integration with object segmentation and detection? To detect drifts of a specific tracker, I propose to analyze its internal state by monitoring its latent parameters over time. For tracker-independent drift detection, I will integrate object segmentation and abjectness (likelihood an image region is an object) measures. To correct drifts, I will integrate object segmentation and object detection (localization) into tracking as drift happens. The use of segmentation is motivated by recent discoveries on the human visual system which seems to rely on spatial and temporal spacing of the objects for effective tracking. I will specifically use object segmentation and localization to explore present-future (or space-time) interaction between video objects. To solve the problem of a variable feature across a video sequence, I will use machine learning, e.g., online multi-kernel learning with support vector machines, to online confirm or reject features. I will handle scale change of objects through monitoring of object states in space and time, through homography transformation, and depth as an added feature. Availability of many datasets, ground-truth data, and metrics of object tracking will enable effective evaluation of the new approaches. I anticipate that developed methods will complement existing online learning approaches, adding an extra level of robustness to tracking-assisted video applications. Interest in widely-applicable (robust) but fast object tracking is large. With my over 20 years of research and industrial experience, I am confident to well advance knowledge and transfer it to related sectors of the Canadian video technology industries.