Transparent Object Tracking with Enhanced Fusion Module
Transparent Object Tracking with Enhanced Fusion Module
复制标题
DOI：
10.1109/iros55552.2023.10341597
复制
发表时间：
2023-09
期刊：
影响因子：
--
通讯作者：
Kalyan Garigapati;Erik Blasch;Jie Wei;Haibin Ling
中科院分区：
文献类型：
--
作者：
Kalyan Garigapati;Erik Blasch;Jie Wei;Haibin Ling
透明物体（如眼镜）的精确跟踪在许多机器人任务（如机器人辅助生活）中起着关键作用。由于这些对象的自适应和反射纹理，传统的跟踪算法，依赖于通用的学习功能遭受性能下降。最近的研究提出了通过融合专用功能将trans-parency意识灌输到现有的通用对象跟踪器中。然而，利用现有的融合技术，新特征的添加导致潜在空间的变化，使得不可能在具有固定潜在空间的跟踪器上结合透明度感知。例如，目前许多基于transformer的跟踪器都是完全预先训练的，对任何潜在的空间扰动都很敏感。在本文中，我们提出了一种新的特征融合技术，将透明度信息集成到一个固定的特征空间，使其在更广泛的跟踪器中使用。我们提出的融合模块，由一个Transformer编码器和一个MLP模块，利用关键查询为基础的转换，嵌入到跟踪管道的透明度信息。我们还提出了一个新的两步训练策略，我们的融合模块，以有效地合并透明度功能。我们提出了一种新的跟踪器架构，使用我们的融合技术，以实现透明对象跟踪的上级结果。我们提出的方法实现了竞争力的结果与最先进的跟踪器TOTB，这是最近发布的最大的透明对象跟踪基准。我们的结果和代码的实施将在https://github.com/kalyan0510/TOTEM上公开。
Accurate tracking of transparent objects, such as glasses, plays a critical role in many robotic tasks such as robot-assisted living. Due to the adaptive and often reflective texture of such objects, traditional tracking algorithms that rely on general-purpose learned features suffer from reduced performance. Recent research has proposed to instill trans-parency awareness into existing general object trackers by fusing purpose-built features. However, with the existing fusion techniques, the addition of new features causes a change in the latent space making it impossible to incorporate transparency awareness on trackers with fixed latent spaces. For example, many of the current days' transformer-based trackers are fully pre-trained and are sensitive to any latent space perturbations. In this paper, we present a new feature fusion technique that integrates transparency information into a fixed feature space, enabling its use in a broader range of trackers. Our proposed fusion module, composed of a transformer encoder and an MLP module, leverages key query-based transformations to embed the transparency information into the tracking pipeline. We also present a new two-step training strategy for our fusion module to effectively merge transparency features. We propose a new tracker architecture that uses our fusion techniques to achieve superior results for transparent object tracking. Our proposed method achieves competitive results with state-of-the-art trackers on TOTB, which is the largest transparent object tracking benchmark recently released. Our results and the implementation of code will be made publicly available at https://github.com/kalyan0510/TOTEM.