多模态目标跟踪是融合多个视觉模态数据实现挑战场景和环境下目标的稳健跟踪。它能够利用模态之间的互补性较好地克服低照度、雾霾和杂乱背景等因素的影响，在视频监控、自动驾驶和机器人等领域有着非常重要的作用。现有多模态跟踪方法一般是用矩形包围盒来表示目标，这样会使得目标模型易受背景信息污染。为此，本项目将研究一种协同图模型，即以图像块为结点构图，利用多模态信息同时优化图像块之间的相似性关系（边权）和图像块表示目标的重要性权重（点权），以此构造目标的协同特征表达，抑制背景影响，实现稳健跟踪。此外，针对非配准视频，本项目将研究一种协同滤波算法，考虑目标在不同模态中的位置和运动关系，协同地实现目标跟踪。

The goal of multimodal object tracking is to track target objects robustly in challenging scenes and environmental conditions by leveraging multiple different yet complementary visual data. It plays a critical role in wide range of applications, such as visual surveillance, self-driving systems and robotics, due to its potentials in handling variety of challenging factors (low illumination, smog, background clutter, etc.). Existing multimodal tracking methods usually localize the object with a bounding box, in which the foreground object trackers/detectors are often disturbed by the introduced background information. To handle this problem, this project aims to learn a collaborative graph with image patches as nodes for multimodal tracking, in which the weight of each node indicates how likely it belongs to the foreground and edges are also weighted for indicating the appearance compatibility of two neighboring nodes. The optimized node weights are then incorporated into the object representation to achieve robust tracking. In addition, we also study a collaborative correlation filter to overcome the dependency of multimodal alignment in conventional methods. The collaborative correlation filter takes the relations of object locations and motions in different modalities into account to track the object collaboratively.