Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition
Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition
复制标题
DOI：
10.1109/tpami.2017.2700390
复制
发表时间：
2018-04-01
影响因子：
23.6
通讯作者：
Tao, Dacheng
中科院分区：
文献类型：
--
作者：
Ding, Changxing;Tao, Dacheng
监控视频中的人脸通常会出现严重的图像模糊、姿态变化和遮挡。在本文中，我们提出了一个基于卷积神经网络（CNN）的综合框架，以克服基于视频的人脸识别（VFR）的挑战。首先，为了学习模糊鲁棒的人脸表示，我们人为地模糊了由清晰静止图像组成的训练数据，以弥补真实世界视频训练数据的不足。使用由静止图像和人工模糊数据组成的训练数据，CNN被鼓励自动学习模糊不敏感的特征。其次，为了增强CNN特征对姿态变化和遮挡的鲁棒性，我们提出了一种Trunk-Branch Entrance CNN模型（TBE-CNN），它从整体人脸图像和围绕人脸组件裁剪的补丁中提取补充信息。TBE-CNN是一种端到端模型，通过在主干和分支网络之间共享低级和中级卷积层来有效地提取特征。第三，为了进一步提高TBE-CNN学习的表示的区分能力，我们提出了一个改进的三重损失函数。系统实验证明了所提出技术的有效性。最令人印象深刻的是，TBE-CNN在三个流行的视频人脸数据库上实现了最先进的性能：PaSC、考克斯人脸和YouTube人脸。在BTAS 2016年视频人物识别评测中，我们也获得了第一名。
Human faces in surveillance videos often suffer from severe image blur, dramatic pose variations, and occlusion. In this paper, we propose a comprehensive framework based on Convolutional Neural Networks (CNN) to overcome challenges in video-based face recognition (VFR). First, to learn blur-robust face representations, we artificially blur training data composed of clear still images to account for a shortfall in real-world video training data. Using training data composed of both still images and artificially blurred data, CNN is encouraged to learn blur-insensitive features automatically. Second, to enhance robustness of CNN features to pose variations and occlusion, we propose a Trunk-Branch Ensemble CNN model (TBE-CNN), which extracts complementary information from holistic face images and patches cropped around facial components. TBE-CNN is an end-to-end model that extracts features efficiently by sharing the low- and middle-level convolutional layers between the trunk and branch networks. Third, to further promote the discriminative power of the representations learnt by TBE-CNN, we propose an improved triplet loss function. Systematic experiments justify the effectiveness of the proposed techniques. Most impressively, TBE-CNN achieves state-of-the-art performance on three popular video face databases: PaSC, COX Face, and YouTube Faces. With the proposed techniques, we also obtain the first place in the BTAS 2016 Video Person Recognition Evaluation.