Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis
Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis
复制标题
用于联合人脸识别和合成的异构人脸可解释解耦表示
DOI：
10.1109/tnnls.2021.3071119
复制
发表时间：
2021
影响因子：
10.4
通讯作者：
Jie Li
中科院分区：
文献类型：
--
作者：
Decheng Liu;Xinbo Gao;Chunlei Peng;Nannan Wang;Jie Li
利用不同的传感器获取不同的人脸信息，更接近真实场景，在生物识别安全领域发挥着重要作用。然而，异构人脸分析仍然是一个具有挑战性的问题，由于不同的模式之间的差异很大。最近的工作要么集中在设计一种新的损失函数或网络架构，直接提取模态不变的功能或合成相同的模态脸最初减少模态差距。然而，前者总是缺乏明确的可解释性，后者的策略固有地带来了综合偏差。在这篇文章中，我们将探索学习复杂异质人脸的简单可解释表示，并同时执行人脸识别和合成任务。我们提出了异构人脸可解释的解纠缠表示（HFIDR），可以显式地解释人脸表征的维度，而不是简单的映射。利用可解释结构，我们可以进一步提取潜在的身份信息进行跨通道识别，并转换通道因子来合成跨通道人脸。此外，我们提出了一个多模态异构人脸可解释的解纠缠表示（M-HFIDR）扩展的基本方法，适用于多模态人脸识别和合成。为了评价其泛化能力，我们构建了一个新的大规模人脸草图数据集。在多个异构人脸库上的实验结果表明了该方法的有效性。
Heterogeneous faces are acquired with different sensors, which are closer to real-world scenarios and play an important role in the biometric security field. However, heterogeneous face analysis is still a challenging problem due to the large discrepancy between different modalities. Recent works either focus on designing a novel loss function or network architecture to directly extract modality-invariant features or synthesizing the same modality faces initially to decrease the modality gap. Yet, the former always lacks explicit interpretability, and the latter strategy inherently brings in synthesis bias. In this article, we explore to learn the plain interpretable representation for complex heterogeneous faces and simultaneously perform face recognition and synthesis tasks. We propose the heterogeneous face interpretable disentangled representation (HFIDR) that could explicitly interpret dimensions of face representation rather than simple mapping. Benefited from the interpretable structure, we further could extract latent identity information for cross-modality recognition and convert the modality factor to synthesize cross-modality faces. Moreover, we propose a multimodality heterogeneous face interpretable disentangled representation (M-HFIDR) to extend the basic approach suitable for the multimodality face recognition and synthesis. To evaluate the ability of generalization, we construct a novel large-scale face sketch data set. Experimental results on multiple heterogeneous face databases demonstrate the effectiveness of the proposed method.