Integrating object segmentation for robust object tracking

集成对象分割以实现稳健的对象跟踪

• 批准号: RGPIN-2017-04801
• 负责人: Amer, Maria
• 金额: $ 1.75万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752022
• 关键词: Integrating; object; segmentation; robust; tracking

Video object tracking is an active research field with applications diverse as augmented reality, self-navigation systems (drones or self-driving cars), human-computer interactions, hyper-linked social video, or athlete performance monitoring. Tracking is performed on raw video signals, captured typically from a single camera; different types of objects can be tracked, such as persons, athletes, vehicles, animals, or ships. Tracking of objects across real-world video signals has many challenges such as variations in object appearance due to scale change, occlusion, articulation, deformation, and interactions of multiple objects. Recent algorithms to object tracking have well advanced the topic in some of these challenges. In this proposal, I will address the following research challenges to advance the state-of-the-arts to single object tracking: 1) drift of object tracking: what to do when a tracker drifts from the target and how to detect such drift? 2) variable object features: how to deal with features, such as color histogram, that significantly vary during tracking? 3) scale change of objects: can we explicitly model scale change or rather implicitly such as through the integration with object segmentation and detection?To detect drifts of a specific tracker, I propose to analyze its internal state by monitoring its latent parameters over time. For tracker-independent drift detection, I will integrate object segmentation and abjectness (likelihood an image region is an object) measures. To correct drifts, I will integrate object segmentation and object detection (localization) into tracking as drift happens. The use of segmentation is motivated by recent discoveries on the human visual system which seems to rely on spatial and temporal spacing of the objects for effective tracking. I will specifically use object segmentation and localization to explore present-future (or space-time) interaction between video objects. To solve the problem of a variable feature across a video sequence, I will use machine learning, e.g., online multi-kernel learning with support vector machines, to online confirm or reject features. I will handle scale change of objects through monitoring of object states in space and time, through homography transformation, and depth as an added feature. Availability of many datasets, ground-truth data, and metrics of object tracking will enable effective evaluation of the new approaches. I anticipate that developed methods will complement existing online learning approaches, adding an extra level of robustness to tracking-assisted video applications. Interest in widely-applicable (robust) but fast object tracking is large. With my over 20 years of research and industrial experience, I am confident to well advance knowledge and transfer it to related sectors of the Canadian video technology industries.

视频对象跟踪是一个活跃的研究领域，其应用范围广泛，如增强现实，自导航系统（无人机或自动驾驶汽车），人机交互，超链接社交视频或运动员表现监控。跟踪是在原始视频信号上执行的，通常从单个摄像机捕获;可以跟踪不同类型的对象，例如人，运动员，车辆，动物或船只。跨真实世界视频信号的对象跟踪具有许多挑战，诸如由于尺度变化、遮挡、接合、变形和多个对象的交互而导致的对象外观的变化。 最近的对象跟踪算法在其中一些挑战中很好地推进了该主题。在这个提案中，我将解决以下研究挑战，以推进最先进的单目标跟踪：1）目标跟踪的漂移：当跟踪器从目标漂移时该怎么办，以及如何检测这种漂移？2)可变对象特征：如何处理跟踪过程中显著变化的特征，如颜色直方图？3)对象的比例变化：我们可以显式地对尺度变化进行建模，或者更确切地说，诸如通过与对象分割和检测的集成来隐式地对尺度变化进行建模吗？为了检测特定跟踪器的漂移，我建议通过监测其潜在参数来分析其内部状态。对于独立于跟踪器的漂移检测，我将集成对象分割和abjectness（图像区域是对象的可能性）度量。为了纠正漂移，我将在漂移发生时将对象分割和对象检测（定位）集成到跟踪中。分割的使用是由人类视觉系统的最新发现所激发的，人类视觉系统似乎依赖于对象的空间和时间间隔来进行有效的跟踪。我将专门使用对象分割和定位来探索视频对象之间的现在-未来（或空间-时间）交互。为了解决视频序列中可变特征的问题，我将使用机器学习，例如，采用支持向量机进行在线多核学习，在线确认或拒绝特征。我将通过监测对象在空间和时间上的状态，通过单应性变换和深度作为附加特征来处理对象的尺度变化。许多数据集、地面实况数据和目标跟踪指标的可用性将使新方法的有效评估成为可能。我预计开发的方法将补充现有的在线学习方法，为跟踪辅助视频应用程序增加额外的鲁棒性。广泛适用（鲁棒）但快速的对象跟踪的兴趣很大。凭借我20多年的研究和行业经验，我有信心很好地推进知识，并将其转移到加拿大视频技术行业的相关部门。