Efficient disentangled representation learning for multi-modal finger biometrics
Efficient disentangled representation learning for multi-modal finger biometrics
复制标题
DOI：
10.1016/j.patcog.2023.109944
复制
发表时间：
2023-09
期刊：
影响因子：
--
通讯作者：
Weili Yang;Junduan Huang;Dacan Luo;Wenxiong Kang
中科院分区：
文献类型：
--
作者：
Weili Yang;Junduan Huang;Dacan Luo;Wenxiong Kang
大多数多模态生物识别系统使用多个设备来捕获不同特征并直接融合多模态数据，同时忽略模态之间的相关信息。本文中手指皮肤和手指静脉图像是从手指的同一区域获取的，因此具有较高的相关性。为了有效地表示数据，我们提出了一种基于分解概念的新型手指解缠结表示学习框架（FDRL-Net），它将每种模态分解为共享和私有特征，从而提高互补性以实现更好的融合，并提取模态不变特征以进行异构识别。此外，为了捕获尽可能多的手指纹理，我们利用三视图手指图像来重建全视图多光谱手指特征，这增加了身份信息和对手指姿势变化的鲁棒性。最后，提出了一种基于 Boat-Trackers 的多任务蒸馏方法，将特征表示能力迁移到轻量级多任务网络。对六个单视图多光谱手指数据集和两个全视图多光谱手指数据集的广泛实验证明了我们方法的有效性。
Most multi-modal biometric systems use multiple devices to capture different traits and directly fuse multi-modal data while ignoring correlation information between modalities. In this paper, finger skin and finger vein images are acquired from the same region of the finger and therefore have a higher correlation. To represent data efficiently, we propose a novel Finger Disentangled Representation Learning Framework (FDRL-Net) that is based on a factorization concept, which disentangles each modality into shared and private features, thereby improving complementarity for better fusion and extracting modality-invariant features for heterogeneous recognition. Besides, to capture as much finger texture as possible, we utilize three-view finger images to reconstruct full-view multi-spectral finger traits, which increases the identity information and the robustness to finger posture variation. Finally, a Boat-Trackers-based multi-task distillation method is proposed to migrate the feature representation ability to a lightweight multi-task network. Extensive experiments on six single-view multi-spectral finger datasets and two full-view multi-spectral finger datasets demonstrate the effectiveness of our approach.