Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking
Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking
复制标题
联合建模运动和外观线索以实现稳健的 RGB-T 跟踪
DOI：
10.1109/tip.2021.3060862
复制
发表时间：
2021
影响因子：
10.6
通讯作者：
Yang Xiaoyun
中科院分区：
文献类型：
--
作者：
Zhang Pengyu;Zhao Jie;Bo Chunjuan;Wang Dong;Lu Huchuan;Yang Xiaoyun
在这项研究中，我们提出了一种新的RGB-T跟踪框架，通过联合建模外观和运动线索。首先，为了获得稳健的外观模型，我们开发了一种新的后期融合方法来推断RGB和热(T)模态的融合权重图。通过离线训练的全局和局部多模态融合网络确定融合权值，然后将RGB和T模态的响应图线性组合。其次，当外观线索不可靠时，我们综合考虑运动线索，即目标和摄像机的运动，使跟踪器具有鲁棒性。我们进一步提出了跟踪器切换器来灵活切换外观和运动跟踪器。在三个最近的RGB-T跟踪数据集上的大量结果表明，所提出的跟踪器的性能明显优于其他最先进的算法。
In this study, we propose a novel RGB-T tracking framework by jointly modeling both appearance and motion cues. First, to obtain a robust appearance model, we develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. The fusion weights are determined by using offline-trained global and local multimodal fusion networks, and then adopted to linearly combine the response maps of RGB and T modalities. Second, when the appearance cue is unreliable, we comprehensively take motion cues, i.e., target and camera motions, into account to make the tracker robust. We further propose a tracker switcher to switch the appearance and motion trackers flexibly. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.