Facial Re-identification Using Deep Learning on Combined Real-Virtual Environments

人脸识别、身份识别和身份验证是生物特征识别领域中常见的问题。多年来，已经开发了许多方法，从而产生了健壮和准确的方法，特别是随着深度神经网络的到来。然而，尽管不断取得进展，算法在扩展到数百/数千用户时仍然受到影响，这些用户具有环境和用户上下文引入的多个可变性来源，例如头部方向、面部饰品、不同背景和使用不同的记录设备。本工作的目的是解决这些限制，并利用深度学习架构开发一种新的统一的特征提取和度量学习框架。通过将人脸识别同化为重新识别，我们的目标是将我们的方法应用到海量数据集，如当前的社交网络或国家警察数据库，在大多数情况下，每个受试者只有几张图像(例如他们的头像或警察的头像照片)。该算法应该能够使用这几张照片在低分辨率和不同视角和姿势的图像质量下识别闭路电视镜头中的对象。在现实世界的监控摄像机上测试这种几乎增强的人脸重新识别范例将是该项目的基本目标。该项目旨在开发一种方法，可以使用警方面部照片中的一到两张图像来识别闭路电视流中的个人。这项工作的目标包括：-实现深度卷积神经网络来处理自动特征提取。-在暹罗网络体系结构中将自动特征提取与度量学习相结合，具体解决验证和重新识别问题。-研究零镜头和一镜头场景下人脸识别的新配置。-扩展重新识别框架，以允许图像到视频的识别，从而能够处理质量较差的闭路电视镜头。-探索稳健的策略，如数据增强和丢弃，以解决面部遮挡和其他可能影响识别的差异来源，例如不同的姿势、衣服、化妆品、眼镜、围巾、帽子等-开发新的深度学习架构，使用来自上下文或受试者档案的语义信息增强视觉面部重新识别。-评估验证与重新识别的性能和局限性，将现实生活中的快照与社交网络档案和/或闭路电视截图联系起来。这项研究对预防犯罪--可以与闭路电视摄像头匹配--人口贩运--非法网站上的照片可以与入境口岸拍摄的图像进行比较--以及提高机场的安全性--将护照照片或可疑监视名单中的图像与机场摄像头进行比较，导致实施更安全和透明的电子闸门。这项研究符合EPSRC人工智能技术和图像与视觉计算的研究领域。

Face recognition, identification and verification are common problems in the field of biometrics. Many approaches have been developed over the years resulting in robust and accurate methodologies, specially with the arrival of deep neural networks. However, in spite of the ongoing progress, algorithms still suffer when scaling to hundreds/thousands of users with the multiple sources of variability introduced by the environment and the user context, such as head orientation, facial accessories, different backgrounds and the use of different recording devices.The aim of this work is to address these limitations and to develop a novel unified framework for feature extraction and metric learning using deep learning architectures. By assimilating face recognition to re-identification, we aim to extend the application of our methodology to vast datasets such as current social networks or national police databases, where in most cases only a few images per subject are available (for example their profile picture or police mugshots). The algorithm should be able to use these few photos to identify subjects in CCTV footage at low resolution and image quality in different view-points and poses. The testing of such a virtually enhanced face re-identification paradigm on real world surveillance cameras will be the underlying objective of this project. The project will aim to develop an approach which can use one or two images from police mugshots to identify individuals in CCTV stream.The objectives of this work include: - To implement deep convolutional neural networks to tackle automatic feature extraction. - To combine automatic feature extraction with metric learning in Siamese network architectures, to specifically address the problem of verification and re-identification. - To investigate new configurations for face recognition in zero-shot and one-shot scenarios. - To extend the re-identification framework to allow image-to-video recognition that's able to tackle poor quality CCTV footage. - To explore robust strategies such as data augmentation and dropout to address facial occlusions and other sources of variation which may affect identification, such as different poses, clothing, cosmetics, glasses, scarfs, hats, etc. - To develop new deep learning architectures that enhance visual facial re-identification using semantic information from the context, or the subject profile. - To evaluate the performance and limitation of verification versus re-identification, to link real life snapshots with social network profiles and/or CCTV captures.This research has potential impact for crime prevention -where mugshots can be matched against CCTV cameras, human trafficking -where pictures in illegal websites can be compared against images taken at the port of entry- and increasing security in airports, -where passport pictures or images in a suspect watch list are compared against the airport cameras, leading to the implementation of more secure and transparent e-gates. This research fits the EPSRC Research areas of Artificial Intelligence Technologies and Image and Vision Computing.